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5.  Introduction  to  Autoregressi ve  Models 

In  the  previous  sections  we  considered  models  for  time  series  in 
which  the  characteristic  and  useful  properties  appropriate  to  the  time 
sequence  were  embodied  in  the  mean  function  f(t)  •,  f(t)  could  be  a 
polynomial  or  a  trigonometric  function.  In  astronomy,  for  example,  it 
is  reasonable  to  suppose  that  the  effect  of  time  is  mainly  in  f(t) 
and  thus  prediction  is  reasonable.  In  economics  and  weather,  for 
example,  the  random  part  u^  is  also  time  dependent,  and  thus  predic¬ 
tion  is  more  difficult.  When  the  effect  of  time  is  embodied  in  u^  , 
we  are  led  to  a  "stochastic  process"  whose  characteristic  properties 
are  described  by  the  underlying  probabilistic  structure.  In  these 
cases,  for  example,  there  are  not  regular  periodic  cycles  but  more  or 
less  irregular  fluctuations  that  have  statistical  properties  of  varia¬ 
bility.  A  process  whose  probability  structure  does  not  change  with 
time  is  called  stationary.  In  Section  5  we  are  mainly  interested  in 
processes  that  are  stationary  or  almost  stationary  or  such  that  at 
least  the  probability  aspect  (as  distinguished  from  a  deterministic 
mean  value  function)  is  roughly  stationary. 

To  illustrate  these  ideas,  let  us  consider  an  autoregressive 
process  of  order  one,  which  is  described  by  the  relationship 

yt  =  ’  t  =  l,2,...  » 

where  the  y.'s  are  observed  values  of  a  random  variable,  and  the  u  's 
are  some  unobserved  random  variables,  called  innovations.  The  innovation 
ut  is  assumed  independent  of  yt_ 1*^-2*’ **  for  values  °f  t  ■ 
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The  distribution  of  y^  and  y ^  is  given  by  the  distribution  of 
y i  and  py^+u^,  and  similarly  the  distribution  of  y^,  y2,  and  y3 
is  given  by  the  distribution  of  y^,  py^  +  Ug,  and  p(py^  +  U2)+u3  . 

Thus  y3  depends  on  y^  »  which  in  turn  depends  upon  y^  ,  and  so  on. 

If  I p ]  <1  ,  then  the  further  apart  the  y's,  the  less  they  are  related. 
An  innovation  u3  is  absorbed  into  y3,  y^,...,  and  thus  the  randomness 
perpetuates  in  time.  We  therefore  say  that  the  effect  of  time  is 
embodied  in  the  u^'s  .  The  above  process  is  pictorially  described  in 
Figure  5.1. 

In  Section  5.1  we  discuss  briefly  some  basic  properties  of 
stochastic  processes  and  introduce  some  notions  which  are  used  subse¬ 
quently. 


5.1  Stationary  Stochastic  Processes 

The  sequence  of  T  observations  which  constitute  an  observed  time 
series  may  often  be  considered  as  a  sample  at  T  consecutive  equally 
spaced  time  points  of  a  much  longer  sequence  of  random  variables.  It 
is  convenient  to  treat  this  longer  sequence  as  infinite,  extending 
indefinitely  into  the  future,  and  possibly  going  indefinitely  into  the 
past.  Such  a  sequence  of  random  variables  y^,  y^ ,...,  or  ...,  -y_2> 
-y_j,  yg,  y^,  y2,...,  is  known  as  a  stochastic  process  with  a  discrete 
time  parameter.  An  objective  of  statistical  inference  may  be  to  deter¬ 
mine  the  probability  structure  of  the  longer  infinite  sequence., 

In  a  stochastic  process  those  variables  that  are  close  together 
in  time  generally  behave  more  similarly  than  those  that  are  far  apart 


1 


2  3  4 


Time  t 


Figure  5.1.  An  illustration  of  the  structure  of 
an  autoregressive  process  of  order  1. 
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in  time.  Usually  some  simplifications  are  imposed  on  the  probability 
structure  of  the  larger  series,  with  the  result  that  the  finite  set  of 
observations  has  implications  for  the  infinite  sequence.  One  simplify¬ 
ing  property  is  that  of  stationarity,  behind  which  is  the  subjective 
idea  that  the  behavior  of  a  set  of  random  variables  at  one  point  in 
time  is  probabilistically  the  same  as  the  behavior  of  a  set  at  another 
point  in  time.  Thus  for  example,  if  the  underlying  probability  struc¬ 
ture  is  assumed  to  be  Gaussian  (normal)  and  stationary,  then  there  is 
one  mean,  one  variance,  and  an  infinite  number  of  covariances.  We  are 
interested  in  finding  out  what  information  about  these  can  be  gleaned 
from  a  finite  number  of  observations. 

A  stochastic  process  y(t)  of  a  continuous  time  parameter  t  can 
be  defined  for  t  >  0  or  -<»  <  t  <  »  .  A  sample  from  such  a  process 
can  consist  of  observations  on  the  process  at  a  finite  number  of  time 
points,  or  it  can  consist  of  a  continuous  observation  on  the  process 
over  an  interval  of  time.  For  example,  the  sample  could  be  a  sequence 
of  consecutive  hourly  readings  of  the  temperature  at  some  location,  or 
it  might  be  a  graph  of  a  continuous  reading.  Often  a  stochastic 
process  with  a  discrete  time  parameter  can  be  thought  of  as  a  sampling 
at  equally  spaced  time  points  of  a  stochastic  process  of  a  continuous 
time  parameter. 


A  discrete  time  parameter  stochastic  process  is  said  to  be 

stationary,  or  strictly  stationary,  if  the  distribution  of  y.  ,„..,yt 

Z1  r 

is  the  same  as  the  distribution  of  yt  +t . yt  +t  for  every  finite 

1  n 

set  of  integers  {tj,...,tn>  and  for  every  integer  t  . 


We  shall  denote  the  mean  or  the  first  order  moment  &y  by  m(t) 
and  the  covariance  or  the  second  order  moment  £(y^  -  m(t) ) (y$  -  m(s) ) 

=  Cov(yt,ys)  by  a(t,s)  .  The  sequence  m(t)  is  arbitrary,  but 
the  second  order  moment  a(t,s)  =  a(s,t)  for  every  pair  s,t,  and  the 
matrix  [cr(t.,t.)],  i,j=l,...,n,  must  be  positive  semidefinite  for 

*  vJ 

every  n  . 

If  the  first  order  moments  exist,  then  stationarity  implies  that 


(5.1) 


&ys  £yt+s  *  s,t  , 


or  that  m(s)  =  m(s+t)  =  m,  say,  for  all  s  and  t.  Stationarity 

also  implies  that  for  all  t>0  (y.  ,y.  )  has  the  same  distribution 

rl  z2 

as  (yt  +t,  yt  +t)  ,  so  that  if  the  second  moments  exist,  then 


Cov(yti*  yt  ) =  =  Cov(ytl+t’  yt2+t^ =  a(ti+t » t2+t)  • 


If  we  set  t  =  -t9  ,  then 


(5.2) 


-  cr( t^-t2 j  0)  a(t1-t2)  ,  say 


Thus  for  a  stationary  process  the  covariance  between  any  two  vari 
ables  yt  and  yt+$  depends  upon  s  ,  their  distance  apart  in  time. 
The  function  a{ s)  as  a  function  of  s  ,  is  called  the  covariance  func 
tion  or  the  autocovariance  function,  and  the  function  of  s 


C°v(yt>  yt+s)  _  a(s) _  _  a(s) 


/ar(yt)/Var(yt+s^  ^a(O)  fa(0)  a(0) 


is  called  the  correlation  function  or  the  autocorrelation  function. 
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A  stochastic  process  is  said  to  be  stationary  in  the  wide  sense  or 
weakly  stationary  if  the  mean  function  and  the  covariance  function 
exist  and  satisfy  (5.1)  and  (5.2).  In  the  case  of  the  normal  distribu¬ 
tion,  weakly  stationary  implies  stricly  stationary  and  vice  versa.  In 
the  general  case,  strictly  stationary  implies  weakly  stationary,  if  the 
second  order  moments  exist. 

5.1.1  Examples  of  Stationary  Stochastic  Processes 

Example  1:  Suppose  that  the  y^'s  are  independent  and  identically 
distributed  with 

o 

(5.3)  eyt  =  m  ,  and  Var(yt)  =  a  ; 
then 

(5.4)  a(t,s)  =  ,  S  =  t  , 

=  0  ,  s  /  t  . 

This  process  is  strictly  stationary;  however,  if  we  drop  the 
requirement  of  identical  distributions,  but  retain  (5.3)  and  (5.4), 
then  the  resulting  process  is  stationary  in  the  wide  sense. 

Example  2.  Suppose  that  the  y^'s  are  identically  equal  to  a 
random  variable  y  with 

2  2 

eyt  =  m  and  Var  yt  =  cx(t,t)  =  a 


Then,  this  process  is  strictly  stationary. 
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Example  3:  Define  a  sequence  of  random  variables  {y^}  as 
follows: 

q 

(5.5)  y.  =  l  (A,  cos  X.t  +  B.  sin  X.t)  ,  t  =  , 

^  j  =  ^  J  J  J  J 

where  the  Xj's  are  constants  such  that  0  <  x^  <  tt  ,  and  A^,...,A  , 
are  2q  random  variables  such  that 


£A,  =  £B.  =  0  , 

vJ  vJ 

■  «j  ■  »J  • 

SA,A.  =  CB  .B.  =  0  , 

*  J  '  J 

£AjBj  =  0  , 


j  l » •  •  •  >  q  > 


j  l,...,q  » 


i^j  ,  1,j-l,...,q,  and 


l » •  •  • » q  • 


Then 


eyt  =  o  , 


and 


eVs 


q  q 

L 


£  £  I  (A-  cos  x.t+B.  sin  x.t)  ( A  -  cos  x.s+B.  sin  X.s) 
i=l  j=l  1  J  J  J  J 

q 

2  2 
l  [£A.  cos  x.t  cos  X.s  +  £B •  sin  X.t  sin  X.s] 
j  —  J  J  J  J  J  J 

a 

p 

l  a. [cos  X.t  cos  X.s  +  sin  X.t  sin  X.s] 
j  —  J  J  J  J  J 

q 

o 

l  a.  cos  X.(t-s)  . 
j  =  l  J  J 
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Since  the  covariance  of  (y^,y  )  depends  only  on  (t-s)  ,  the 

distance  between  the  two  observations,  and  since  =  0  for  all  t, 

the  sequence  {y^.}  is  stationary  in  the  wide  sense.  If,  however,  the 

A.'s  and  the  B.'s  are  also  normally  distributed,  then  the  y.'s  will 
J  J  ^ 

also  be  normally  distributed,  and  then  the  process  will  be  stationary  in 
the  strict  sense. 

The  point  of  this  example  is  that  every  weakly  stationary  process 
can  be  approximated  by  a  linear  combination  of  the  type  indicated  by 

(5.5) . 

Example  4;  Let  . ,.,v_^,  Vg,  v^,...  be  a  sequence  of  indepen¬ 
dent  and  identically  distributed  random  variables,  and  let  aQ,  a^..., 
aq,  be  q+1  coefficients.  Then 

(5.6)  y^  =  oigV^.  +  a^v^  i  +  . . .  +  aqvt  q»  t  =  . . .  ,-l ,  0,1,...  , 

2 

is  a  stationary  stochastic  process.  If  £v^  =  y  ,  and  Var  v^  =  a  , 
then 

eyt  =  Y  (ag  +a^  +  ...  +aq) 

and 

2 

Cov( y^ ,  )  -  ct  (^g1^ g  +  ...  ,  s  -  0, . . .  ,q  , 

=  0  ,  s  =  q+1, . . .  , 

and  so  Ty t }  is  weakly  stationary.  Thus,  for  {y^}  to  be  weakly 
stationary,  all  we  need  is  that  the  v^'s  have  the  same  mean,  the  same 
variance,  and  that  they  be  uncorrelated. 

The  process  (5.6)  is  known  as  a  finite  moving  average. 
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The  infinite  moving  average 


(5.7) 


yt  =  l 

z  s=0 


“svt-s 


means  that  the  random  variable  y^  ,  when  it  exists,  is  such  that 

n  ? 

(5.8)  lim  £(yt-  l  asvt-s)  =  0  * 

n->- a>  s=0 

A  sufficient  condition  for  the  existence  of  y^  is  that  the  v^'s 
be  uncorrelated  with  a  common  mean  (=0)  and  variance,  and 

(5.9)  l  a2  <  »  ; 

s=0  s 

see  Anderson  (1971),  p.377. 

co 

When  (5.8)  holds,  the  infinite  sum  £  a  v,  is  said  to 

s=0  s  t_s 

converge  in  the  mean  or  in  the  quadratic  mean . 


6.  Basic  Notions  of  Multivariate  Normal  Distributions 

Two  random  variables  X  and  Y  with  means  ux  and  y and 
2  2 

variances  ctx  and  a ,  respectively,  are  said  to  have  a  bivariate 
normal  distribution,  or  a  bivariate  Gaussian  distribution,  if  their 
joint  density  function  is  given  by 


x  y  xy 


f(x,y)= 


JTJ 


exp 


2(1-p1) 


x-yY  o  y-uv  o  x-y  y-y 

( — ^-)2+( — £)  Z-2P  ( — -){ — X-) 

°x  ay  W  *x  °y 
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e(X-yx)(Y-u  ) 

then  p  = - —  is  the  correlation  between  X  and  Y  ; 

xy  a  a  - 

J  x  y 

-°°<X<°°,  -co  <  y  <  . 

The  marginal  density  of  X  is  given  by 


f(x) 


exo 


1  x“px  2 

If _ I) 

2(  Px  } 


-00  <  X  <  00  . 


This  is  the  normal  density  function,  which  we  will  henceforth  denote  by 
n(x|V  a2)  . 

Similarly,  the  density  function  of  Y  is  also  normal, 

n(yl*y  ay)  • 

We  can  show  (Anderson  (1984),  p.37),  that  f(x|y*),  the  conditional 
density  function  of  X  ,  given  Y  =  y*  is  also  normal,  but  with  a  mean 

ax  2  2 

ux  +  pxy  a~  (y*~V  5  and  variance  ax^1_pxy^  *  That  is’ 


f(x|y*)  =  n(xlpx+pxy  (y*‘V  »  ax^1-pxy^  * 


r 


xy' 


Thus,  the  variance  of  X  given  that  Y =y*  does  not  depend  on  y*  , 
and  its  mean  is  a  linear  function  of  y*  . 

The  mean  value  of  a  variate  in  a  conditional  distribution,  when 
regarded  as  a  function  of  the  fixed  variate,  is  called  a  regression. 

Thus,  the  regression  of  X  in  the  situation  above  is 

o 

ii  +  p  —  ( y*-u  )  . 

^x  xy  Cy  My' 

The  tri vari ate  normal  distribution  of  three  random  variables  X,Y, 


and  Z  is  defined  in  a  manner  akin  to  the  bivariate  normal  distribution, 
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once  the  means  u  ,  y  ,  and  p 
x  y 


2  2 

,  the  variances  a  ,  a  , 
z  x  y 


2 

V 


respectively,  and  the  correlations  between  their  pairs  p  ,  p  ,  and 

xy  xz 

PyZ  ,  are  specified.  Let  f(x,y,z)  denote  the  joint  density  function 
of  this  trivariate  normal  distribution;  let  f(x,y]z)  denote  the  joint 
density  function  of  X  and  Y  conditional  on  Z  =  z  .  Then 


f(x,y |z) 


f(x,y,z) 

f(z) 


where  f(z)>0  is  the  marginal  density  function  of  Z,  which  is  again 
normal.  A  property  of  the  normal  distributions  is  that  f(x,y|z)  is  a 
bivariate  normal  density  (Anderson  (1984),  p.37). 

Let  f(x|z)  and  f(y|z)  denote  the  marginal  densities  of  X  and 
Y  conditional  on  z  ,  respectively;  these  densities  can  be  obtained  via 
f(x,y|z)  .  Let  £(X|z)  and  e(Y|z)  denote  the  expected  values  of  X 
and  Y  conditional  on  z  ,  respectively.  From  our  previous  discussions 
on  the  bivariate  case,  we  recall  that  f(x|z)  and  f(y|z)  are  also 
normal,  and  that  e(X|z)  and  e(Y|z)  can  be  written  as 

£(X  |z)  =  a  +  3z  ,  and 
£(Y|z)  =  y  +  Sz  . 

The  correlation  between  X  and  Y  conditional  on  z  ,  denoted  by 

p . .  is  called  the  partial  correlation  between  X  and  Y  when  Z 

xy*  z 

is  held  constant. 

Thus,  we  have 

=  fi(X-(a+3z) ) g(Y-(y+6z) ) 

-(a+ez))2e(Y-(Y+6z))2 
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Small  values  of  p  imply  that  there  is  little  relationship 
xy  •  z 

between  X  and  Y  that  is  not  explained  by  Z  .  We  can  also  verify 
[Anderson  (1984),  p.4l)  that 


pxy~pxz  pyz 


xy*z 


^“pyz 


To  discuss  the  idea  of  the  "multiple  correlation"  between  X  and 
the  pair  (Y,Z)  ,  let  us  denote  by  e(X|y,z)  the  expected  value  of  X 
conditional  upon  Y  =y  ,  and  Z  =  z  .  Again,  from  our  discussion  of  the 
bivariate  case,  we  note  that  e(Xjy,z)  can  be  written  as 


e(x|y,z)  =  a  +ey  +yz 


where  a,  e,  and  y  are  constants. 

Now  let  us  consider  the  correlation  between  X  and  an  arbitrary 
linear  combination  of  Y  and  Z,  say  bY  +  cZ,  where  b  and  c  are 
arbitrary  constants. 

2 

Then,  the  multiple  correlation  between  X  and  (Y,Z)  ,  say  R  is 

=  max[Correlation(X,  (bY+cZ))]^  . 
b,c 


It  turns  out  that  the  values  of  b  and  c  are  0  and  y  respectively. 
Thus,  the  multiple  correlation  is  the  correlation  between  X  and  eY+yZ  . 


7.  Estimation  of  the  Correlation  Function 

One  of  the  first  steps  in  analyzing  a  time  series  is  to  decide 
whether  the  observations  y^,  y2,...,yy  are  from  a  process  of 
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independent  random  variables  or  from  one  in  which  the  successive 
variables  are  correlated.  If  the  process  is  assumed  stationary,  then 
r(h),  an  estimate  of  the  correlation  function,  enables  us  to  infer  the 
nature  of  the  joint  distribution  that  generates  the  T  observations. 

To  see  this,  consider  a  pair  of  random  variables  and  Yt+k’ 

separated  by  some  lag  k  ,  where  k=l,2 .  The  nature  of  their 

joint  probability  distribution  can  be  inferred  by  plotting  a  "scatter 
diagram"  using  the  pair  of  values  y^  and  yt+k  ,  for  t=l,2,. ..  ,T-k  . 

In  Figure  7.1  we  show  a  scatter  diagram  for  and  Y^+k  ;  this 

diagram  indicates  that  a  large  value  of  Y^  tends  to  lead  us  to  a 

large  value  of  Yt+k  ,  and  vice  versa.  When  this  happens,  we  say  that 
Yt  and  Yt+k  are  positively  correlated.  In  Figure  7.2  the  scatter 
diagram  shows  that  a  large  value  of  Yt  leads  us  to  a  small  value  of 
Yt+k  and  vice  versa;  in  this  case,  we  say  that  Y^  and  Yt+k  are 
negatively  correlated.  A  key  requirement  underlying  our  ability  to 
plot  and  interpret  the  scatter  diagram  is  the  assumption  of  stationarity. 
Because  of  this  assumption  the  joint  distribution  of  Y^  and  Y^+k  is 
the  same  as  the  joint  distribution  of  any  other  pair  of  random  variables 
separated  by  a  lag  of  k  ,  say  Yt+S  and  Y.j.+s+k  ,  for  some  s^O  . 

A  formal  way  of  describing  the  impressions  conveyed  by  a  scatter 
plot  is  via  an  estimate  of  the  correlation  function;  this  estimate  is 


also  known  as  the  serial  correlation.  If  the  observations  y^yg,..., 
yT  are  assumed  to  be  generated  by  a  process  with  mean  0  ,  then  r*(l)  , 
the  first  order  serial  correlation  coefficient  is  defined  as 
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Figure  7.1.  Scatter  plot  of  Yt  and  Yt+k  showing  a 
positive  correlation  between  the  variables. 


Figure  7.2.  Scatter  plot  of  Yt  and  Yt+R  showing  a 
negative  correlation  between  the  variables. 
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(7.1) 


r*(l)  = 


T-l 

t4  *** 

I  y2t 

t=i  1 


If  the  mean  of  the  process  is  not  known,  (7.1)  is  modified  by 
replacing  yt  and  yt  ^  by  the  deviation  of  these  from  the  sample 


mean 


T 

y  ,  where  y  =  l  y,/T  .  Thus  we  have 
t=l  1 


(7.2) 


T-l 


KU  =  l  (yt-y)(yt+1-y)/  l  (yt-y)' 

t=i  r  t=i  z 


Higher  order  serial  correlations  are  similarly  defined;  for 
example,  r*(h)  ,  the  h-th  order  serial  correlation  is 


(7.3) 


r*(h)  = 


T-h 

th y*  ytth 
T 

I  y2t 

t=i  r 


or  in  analogy  with  (7.2)  it  is 


(7.4) 


r(h)  = 


I  (yt-y)‘ 


t=i 


-16- 


8.  Autoregressive  Processes 

One  of  the  simplest,  and  perhaps  the  most  useful,  stochastic 
process  which  is  used  to  model  a  time  series  is  the  autoregressive 
process.  A  sequence  of  random  variables  y^,y is  said  to  be  an 
autoregressive  process  of  order  p  ,  abbreviated  as  AR(p)  ,  if  for 
some  constant  y  and  integer  p 

(8.1)  (yt-u)  +e1(yt_1-y)  +  ...+  sp(yt_p-y)  =  ut ,  t  =  p+l,p+2,...  , 

with  u  +2»...»  being  independent  and  identically  distributed 

2 

with  mean  0  and  variance  a  ,  and  u^  independent  of  y.  y. 

...  .  We  shall  set  y=0  in  the  following  discussion.  The  random 

variable  ut  is  called  an  innovation  or  a  disturbance.  We  shall  refer 

to  the  sequence  {u^}  as  an  innovation  process . 

It  is  convenient  to  generalize  (8.1)  to  a  doubly  infinite  sequence 

...,  y_j,  yQ,  y^,...,  resulting  in  a  doubly  infinite  sequence  ...,u_^ 

Uq,Uj,...  .  Such  processes  are  also  known  as  autoregressive  processes 

k  dcf 

If  we  use  the  foreward  lag  operator  p  ,  where  p  u^  =  u^  for 
any  integer  k  ,  then  (8.1)  can  also  be  written  as 

(8.2)  (p^  + ^  +  ...  +  3pp^)y^._p  =  u^.  . 

Since  Ayt=yt+1  -  yt  =  pyt  -  yt  =  (p-l)yt  >  we  have  the  result 
that  A  =  p-1  ;  recall  that  A  is  the  foreward  difference  operator 
introduced  in  Section  3.3.  Thus  we  may  say  that  the  operator  acting  on 
y.  can  also  be  written  as  a  polynomial  in  A  of  degree  p  .  If 

L-p 

Bp  t  0  ,  then  the  left  hand  side  of  (8.2)  can  be  written  as  a  linear 

combination  of  y^.  ,  Ayfp’  A  -^t-p’ *  ’  *  ,A^t-p  anc*  1S  therefore  called 

a  stochastic  di  ff erence  equation  of  degree  £  . 
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Unless  otherwise  stated  (see  for  instance  Section  8.9),  we  shall 
assume  that  the  stochastic  process  described  by  (8.2)  is  stationary. 

In  Section  8.1  we  shall  determine  the  conditions  under  which  u^  is 
independent  of  y.^,  yt-2’*'*5  * 

The  model  (8.1)  can  be  used  to  generate  other  processes.  For 
example,  should  we  want  to  incorporate  the  effect  of  a  trend  in  (8.1), 

then  we  add  to  the  left  hand  side  of  (8.1)  the  term  }*  ,  where 

the  z..  's  are  known  functions  of  time;  this  matter  is  discussed 
further  in  Section  8.7. 

Autoregressive  processes  were  suggested  by  Yule  (1927),  and  were 
applied  by  him  to  study  sunspot  data.  Gilbert  Walker  (1931)  extended 
the  theory  and  applied  it  to  atmostpheric  data.  In  what  follows  we 
shall  study  the  structure  of  autoregressive  processes,  and  address  the 
related  questions  of  inference  and  prediction. 

8.1  Representation  as  an  Infinite  Moving  Average 

If  we  inspect  (8.1),  we  see  that  y^  is  expressed  as  a  linear 

combination  of  the  previous  y^'s  and  u^  .  We  shall  now  study  the 

conditions  under  which  y^  can  be  written  as  an  infinite  linear  com¬ 
bination  of  u^  and  the  earlier  ur's  .  To  see  the  idea,  we  consider 
an  AR(1)  process 

yt  -  pyt_!  +  ut  , 

and  note  that  since  y^_^  =  py^  +  ,  we  have 

t-l  +  p2yt-2  ' 


yt  =  ut  +  pu 
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Successive  substitution  of  the  type  indicated  above  ieads  us  to 


wri  te 


(3.3)  yt  -  ut  +  put.j  +  P2ut.2  +  ...  +  Psut_s  +  ps+1y 


t- ( s+1 ) 


so  that 


S  c  +  1 

(8.4)  yt  "  (ut  +  put-l +  • ‘ *  +  p  ut_s)  =p  yt-(s+l) 


The  difference  between  y^  and  a  linear  combination  of  the 

p  ^.1 

(s+1)  ur's  is  therefore  p  yt  ,  and  this  becomes  small  when 

| p  |  < 1  and  s  is  large.  In  particular 

(8.5)  £[yt  "  (ut  +  put-l  +  * *.  + p  Ut-s) 1  =  p^  ^£yt-(s+l) 

will  not  depend  on  t  ,  if  we  assume  that  (yt }  is  a  doubly  infinite 
stationary  process.  As  s  increases,  (8.5)  will  go  to  0  ,  and  so  we 
can  write 

oo 

v  r 

yt  =  l  p  u 

r  r=0  z  r 


and  say  that  the  infinite  sum  on  the  right  of  the  above  equation 
converges  in  the  mean  to  yt  .  (See  Section  5.7.) 

Let  us  now  consider  an  AR(p)  , 

P 

4o  Vt-r  =  ut  >  B0  ■  1  • 

so  that 

yt  =  ut  Slyt-1  "  32yt-2 ~  **•  “  epyt-p  ' 
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Replacement  of  t  by  t-1  yields 

yt-l  =  ut-r  elyt-2  "  32yt-3  "  •••  "  epyt-p-l  * 
which  upon  substitution  gives 

yt  =  ut  ~  3l^Ut-r3lyt-2“32yt-3  "  epyt-p-l^"e2yt-2“  *  ‘  ‘  "Bpyt-p 

=  ut_  3lut-l  "  ^2  "  el^yt-2  "  ***  +  3lBpyt-l-p  ‘ 

Continuing  in  the  above  manner  s  times,  we  arrive  at 
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is  a  set  of  recursion  relationships  for  the  coefficients.  Continuation 

•k 

of  this  procedure  leads  us  to  write,  for  <5q  =  1  , 

(s-7)  yt  =  l  <  Vi 

t  i=0  i  t  i 

if  the  infinite  sum  on  the  right-hand  side  of  (8.7)  converges  in 

the  mean  to  y^  .  We  shall  next  see  the  conditions  for  this  convergence. 


8.1.1  Conditions  for  Convergence  in  the  Mean  of  Autoregressive 
Processes 

The  material  of  Section  8.1  can  be  formalized  by  using  the  backward 
lag  operator  x,  where  xyt  =  y^_^  ,  and  writing  the  process  (8.1)  as 

P 

l  er/yt  =  u  • 
r=0  r  z  z 

Then,  formally  we  can  write  our  AR(p)  process  as 


where 


P 

y f  =  (  l  fycr)_1  u.  , 

z  r=0  r  r 


(  I  3  Xr)_1 

r=0  r 


the  6^'s  are  the  coefficients  in  the  equality 
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(8.8)  (  l  0  z r)_1  =  l  5Zr 

r=0  r  r=0  r 

on  the  basis  that  the  above  equality  can  be  so  written  meaningfully. 

It  can  be  verified  (Anderson  (1971),  p . 169 )  that  the  6  1 s 

of  (8.8)  are  indeed  the  same  as  the  6^‘s  of  (8.7),  which  we  recall 

★ 

were  obtained  by  successive  substitution;  thus  we  write  6=6. 

r  r 

In  order  to  see  the  conditions  under  which  it  is  meaningful  to 
write  (8.8),  we  consider 


(8.9) 


BqxP  +  ^x13"1  + 


V 


=  0 


the  associated  polynomial  equation  of  the  stochastic  difference  equa¬ 
tion  (8.1)  (our  AR(p)  process). 

For  0p^O,  let  Xj,...,x  be  the  p  roots  of  (8.9).  If 
|x.  |  <  1  ,  for  i  =  l,...,p,  then  it  is  clear  that  Zj,...»Zp>  the  roots 


I 

r=0 


are  such  that  =  1/x^  and  that  |z^|  >1  .  Now,  for  any  z  such 
that  | z I  <  min|z-|,  the  series 


P 


1 


p 

n  (1-  -j-) 
i=l  *i 


P 

n 

i=l 


(8.10) 
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converges  absolutely.  Thus  we  see  that  when  x^,...,x  ,  the  roots  of 

the  associated  polynomial  equation  of  an  AR(p)  process,  are  less 

p 

than  1  in  absolute  value,  we  can  write  (  £  3  zr)"  =  I  6  zr  . 

r=0  r  r=0  r 

To  argue  convergence  in  the  mean  of  the  AR(p)  process,  we  con- 


r 

sider  the  expression  (  £  B  zr)”^  and  note  that  by  a  formal  long 


r=0 


hand  division 
1 


l+e-iz  1-. .  .+b„z 


1  -■ 


^zfB2Z2+...+BpzP 


r,  ,  #rp  /  1+3jZ+B2Z  +..«+BpZ 

,2x2 


(B J,-Bf)z^+...+(6n-B1en  i  )z "-Bi 3nz 

=  1  -  B,Z  -  •  - P-  - ^L-R- 


p+1 


l+B-jZ+BgZ^. .  .+BpZP 


If  we  continue  in  the  above  manner,  we  see  that 


l+BjZ+.-.+BpZ 


,S+1.  _l„  s+p 

2  s  “slz  z 

-  =  l+(S1Zt69Z£+...+6  zb  +— - =*- - 

P  1  2  5  l+8jZ+...+6pZP 


where  the  6  's  and  the  a  .'s  satisfy  the  same  recurrence  relation- 
r  si  J 

*  1  *  .  * 

ships  as  the  6  's  and  the  a  .  s  of  Section  8.1.  Thus  6=6 
r  si  r  r 

•ft 

and  a  ..  =  a  .  .  (See  8.6.) 

In  view  of  (8.10),  we  now  see  that 


,S  +  1.  TS  +  P 

a.z  +. . . +a ^  z 
Si _  sp 

1+BjZ  ..+BpzP 


must  con¬ 


verge  to  0  for  jz  |  <  mi n  |z  .  |  ,  and  in  particular  for  z  =1  .  This 

i  1 

implies  that  the  a^.  0  (as  s ->■«>)  for  each  i  .  Thus,  if  {y^}  is 
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a  stationary  process 

s 

£^yt"rJ0  6rut-r^  =  £^aslyt-s-l+‘*,+aspyt-s-p^ 

will  not  depend  on  t  and  will  converge  to  0  as  s  -*-«>  . 

We  therefore  have 

co 

yt  r^g  Vut-r 

in  the  sense  of  convergence  in  the  mean.  We  have  proved 

P 

Theorem  8.1:  If  the  roots  of  the  polynomial  equation  J  3„xp“r  =  0 

r=0  r 

P 

associated  with  a  stationary  AR(p)  process  £  &  y .  =  u  are 

r=0  r  z~r  z 

less  than  1  in  absolute  value,  then  yt  can  be  written  as  an  infinite 
linear  combination  of  u^.,  u^,  • 

Note  that  whenever  y^  can  be  written  as  an  infinite  linear  combi¬ 
nation  of  u^,  ut_p...,  yt  will  be  independent  of  the  future  innova¬ 
tions  ut+p  ut+2,...,  ;  this  follows  from  our  assumption  that  the 
sequence  of  innovations  {u^ }  is  mutually  independent.  We  thus  have 
as  a  corollary  to  Theorem  8.1 

Corollary  8.2:  If  the  roots  of  the  polynomial  equation  associated  with 
a  stationary  AR{p)  process  are  less  than  1  in  absolute  value, 
yt  is  independent  of  u^,  ut+2,...,  . 
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8.2  Evaluation  of  the  Coefficients  6^  and  their  Behavior 

Suppose  that  the  roots  of  the  associated  polynomial  equation 
P 

l  6  xp'r  =  0  ,  are  less  than  1  in  absolute  value.  Then,  by 
r=0  r 

Theorem  8.1,  we  can  write 

co 

yt  =  l  6  u  , 

z  r=0  r  z  r 


where  the  6^'s  are  to  be  viewed  as  weights  associated  with  the  present 
and  past  innovations  u^,  ut_j,...  .  Our  goal  is  to  determine  a  proce¬ 

dure  by  which  the  <5r's  can  be  expressed  in  terms  of  the  known  Br's  , 
and  also  to  see  if  there  is  any  discernable  pattern  in  the  6r's  .  Such 
a  pattern  will  enable  us  to  interpret  the  behavior  of  our  sequence  {y^}. 

From  (8.10)  we  note  that  since 


p 

(  l 


^  3sz‘ 

r=0  r  s=0  s 


P 

l 


=  (  I 

r=0 


V 


*)  l 

s=0 


esz' 


or  that 


l 


P 

l 


r=0  s=0 


es8rzr+s  =  i 


Replacing  r  by  (t-s)  and  by  suitable  re-arrangements ,  we  have 


P-1 


l  (  l  Ps6t.s)zt  +1(1  3  6  )zt  =  1 

t=0  s=0  s  L  s  t=p  s=0  s  r  s 


which  is  an  identity  in  z  (the  series  converging  absolutely  for 
|z  |  <  1)  .  An  inspection  of  the  above  reveals  that  the  coefficient  of 
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z®  on  the  left  hand  side  is  1  and  the  coefficients  of  the  other 
powers  of  z  are  zero;  thus  we  have  the  following  set  of  relationships 
between  the  6's  and  the  3 1 s  : 


(8.11) 


eo5o  50  ~  *  5 

Vi  +  Bi6o =  5i  +  3i =0  » 


305p-l  +  • • •  + Bp-150  = 0  »  and 


(8.12)  e06t  +  +  Bp6t-p  =  0  ’  t  =  p,  p+1, . 

We  note  that  (8.12)  is  a  homogeneous  difference  equation  which 
corresponds  to  the  (stochastic)  difference  equation  that  describes  the 
AR(p)  process 

B0yt  +  Blyt-1  +  *  *  *  +  Bpyt-p  =  ut  ’  Bo  =  1  • 

If  the  roots  of  (8.9),  the  polynomial  equation  associated  with  an 
AR(p)  process  are  distinct,  then  the  general  solution  of  (8.12)  is  of 
the  form 

P 

(8.13)  5  -  I  k.x.  r  =  0,1 . 

r  i=l  11 

where  k,,...,k  are  coefficients. 

1  p 
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If  a  root  is  real,  then  the  coefficient  is  also  real. 

If  a  pair  of  roots  and  x^.+1  are  conjugate  complex,  then  k.  and 
k_.+^  are  also  conjugate  complex  and  k^x^  +  k.^x^  is  real, 
r  =  0,1,. . .  . 

Equations  (8.11)  give  us  the  boundary  conditions  for  solving  (8.12) 
The  p  equations  (8.11)  enable  us  to  determine  the  p  constants 
k.,...,k  by  substituting  (8.13)  in  (8.11). 

r 

The  above  material  can  be  better  appreciated  via  some  special  cases 
these  are  discussed  below. 

8.2.1  Special  Cases  Describing  the  Evaluation  and  Behavior 
of  6  's 

We  shall  consider  here  two  examples,  an  autoregressive  process  of 
order  1  and  an  autoregressive  process  of  order  2. 

An  Autoregressive  Process  of  Order  1 

Suppose  that  in  (8.1)  p  =  l  (and  y=0),  so  that  an  AR(1)  process 
is 

e0yt  +6lyt-l  =  ut  »  for  t  =  2>3>---  • 

The  associated  polynomial  equation  for  the  above  process  is 

6gX  +  gjX0  =  0  , 

and  so  with  3q  =  1,  x  =  -e^  is  the  only  root. 
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From  (8.11)  we  have  6q  =  1  and  6^  =  -3^  »  so  that  the  coefficient 
in  (8.13)  is  1  .  Thus,  for  our  AR(l)  process  the  coefficients 
6^  are  such  that 

(8.14)  6r  =  k1xi  =  (-f1)r  •  ' 

Now,  if  we  assume  that  the  process  is  stationary,  then  in  order  to 
be  able  to  write  yt  as  an  infinite  linear  combination  of  u^,  u^,..., 
we  need  to  have,  by  Theorem  8.1,  jx|  <1  or  equivalently  |g.|  <1  .  . 
Thus,  when  |g^|  <1  ,  we  can  write 

oo 

(8.15)  yt  =  I  «rut-r  • 

z  r=0  r  r  r 

We  note  from  (8.14),  that  the  weights  6r  exponentially  decay  in 
r  when  1 6-^  |  <1  .  The  decay  is  smooth  if  3^  <  0  ,  and  the  decay 
alternates  in  sign  if  3^>0  .  This  behavior  of  the  s^'s  implies  that 
in  (8.15)  the  remote  innovations  receive  smaller  weights  than  the  more 
recent  ones.  Such  results  are  useful  for  explaining  the  behavior  of 
the  series  y^,  yt_p...,  and  also  interpreting  forecasts  in  autoregres¬ 
sive  processes  or  order  1. 

An  Autoregressive  Process  of  Order  2 

Now  suppose  that  in  (8.2)  p  =  2  (and  u  =  0)  ,  so  that  an  AR(2) 
process  is 

u  *  t  —  3,4,...  . 


Wt  +  Vt-1  +  e2yt-2 
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With  3q =  1  the  associated  polynomial  equation  for  our  AR(2)  process 
becomes 

2  0 
x  +  gjX  +  $2X  =  0  • 

If  x1  and  are  the  roots  of  the  above  equation,  then  x^  =  (-Bj  ± 
-4e2  )/2  ,  i  =1,2  . 

2 

If  the  roots  x^  and  x2  are  real  and  distinct,  that  is  Bi>4b2» 
then  (8.11)  and  (8.12)  give  1  =  k-^x8  +  k2x8  =  kj  +  k2  and  kixj  +  k2x2 
=  -Bj  =  x ^  +  x2  •  The  solution  is 


-x. 


kn  = - 

1  X,  -X 


and 


1  2 


(/  =:  _ _ _ 

K2  x1-x2 


Then 


xr+1  -  xr+1 

^8-16^  5r  =  1*1-*2  *  r  =  0*1»2 . 

If  we  assume  that  our  AR(2)  process  is  stationary,  then  in  order 
to  be  able  to  write  yt  in  the  form  (8.15),  that  is,  as  an  infinite 
linear  combination  of  u^,  ut_^,...,  we  need  to  have  (by  Theorem  8.1) 
|x.|  <1  ,  i  =  1,2  .  This  in  turn  implies  that  the  coefficients  3^  and 
32  will  have  to  satisfy  the  following  conditions: 

&1  +  $2  > 

(8.17)  3^  -  B2  <  1  >  and 


-1  <  b2  <  1  . 
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The  above  conditions  define  a  triangualr  region,  shown  in  Figure 
8.1,  in  which  the  coefficients  and  ^  must  Tie;  also  see  Box  and 
Jenkins,  (1976),  p.59. 

When  |x^  |  < 1  ,  i  = 1,2,  and  x^  and  are  real ,  that  is,  when 

3^  and  ^  ^ie  outside  the  parabolic  region  of  Figure  8.1,  then  from 

(8.16)  it  is  clear  that  the  weights  5  are  a  linear  combination  of 

r+1  r+1 

two  exponentially  decaying  functions  of  r,  x^  and  x ^ 

When  | x^  |  <  1 ,  i=l,2,  and  when  x^  and  x^  are  complex,  that 
2 

is  <  4^2  so  that  and  e2  "*1*e  ”'n  t^ie  Para^)0"*1'c  region  of 

■j  0  ^  ^  Q 

Figure  8.1,  x^  and  X2  may  be  written  as  x^  =  ae  and  X2  =  ae" 
where  i  =  \Pl  ;  since  | x^  |  <1  and  |x2|  <  1  ,  a  <  1  .  Thus 


so  that 

(8.18) 


k„  =  — r 


'1  Je  Q-ie 

e  -e 


and  k„  = 


-e 


-10 


2  “  Qie  -ie  ’ 

e  -e 


6  =  k-x*!'  +  k0Xo 
r  11  2  2 


=  a 


r  ei e (r+1)  _  e-1e(r+l) 


10  -10 

e  -  e 


=  a 


r  sin(e(r+l) ) 


sin  0 


1  0 

since  e  =  cos  0  +  i  sin  0  . 

Thus  <5^  is  a  damped  sine  function  of  r  ,  whose  nature  is  illus¬ 
trated  in  Figure  8.2.  Such  a  damped  sinusoidal  behavior  of  the  weights 
offers  an  explanation  of  an  oscillatory  pattern  of  the  y^'s  often 
observed  in  otherwise  nonperiodic  stationary  time  series.  (Also  see 
Section  8.6.) 


3]^  and  32 


Weights  & 
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Figure  8.2.  Behavior  of  the  weights  <5r  as  a  function  of  r  , 
for  an  autoregressive  process  of  order  2  whose 
associated  polynomial  equation  has  complex  roots. 
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In  conclusion,  we  note  that  for  a  stationary  autoregressive  process 
of  order  2,  the  remote  innovations  in  a  (8.15)  type  representation  of 
the  series  receive  a  smaller  weight  than  the  more  recent  ones,  regardless 
of  whether  the  roots  of  the  associated  polynomial  equation  are  real  or 
complex.  The  nature  of  the  roots  determines  whether  the  weights  decay 
exponentially  or  sinusoidally. 

8.3  The  Covariance  Function  of  an  Autoregressive  Process 

If  the  joint  distributions  of  the  y^'s  are  normal,  then  the 

process  is  completely  determined  by  its  first  and  second  order  moments, 

2 

£yt  ,  £yt  ,  and  £ytyt+s  ,  s  =  l,2,...,  .  If  the  joint  distributions  are 
not  normal,  the  above  moments  still  give  us  some  information  about  the 

2  2 

££YS  »  the  correlation  between  yt  and 
yt+s  (assuming  that  £yt  =  0  for  all  t)  ,  is  a  measure  of  the  relation¬ 
ship  between  the  two  variables  y^  and  yt+s  for  t  =  l,2,... 

If  the  process  is  stationary,  then  all  the  variances  are  the  same, 
and  the  covariances  depend  only  on  the  difference  between  the  two  indices. 
Thus 

eVt+s  =  o(s)=o(-s)  ,  S  =  ....  -1,  0,  +1 . 

Recall  that  a(s)  is  also  called  the  autocovariance  function  and  that 
a(s)/a(0)  is  also  called  the  autocorrelation  function;  it  will  be  denoted 
by  p(s)  and  abbreviated  as  ACF. 

We  shall  now  look  at  the  properties  of  the  covariance  function  a(s)  . 


process.  For  example. 


eVt+ 
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If  we  replace  t  by  t  -  s 
P 

l  Mt-r  =  Ut  »  we  have 
r=0  r  z  r  r 


in  y.  =  l  6  u.  and  multiply  it  by 
1  q=0  q  T'"q 


(8.19) 


P 

4o  ^ryt-ryt-s 


q=0 


6qUt-s-qUt 


Now  £yt_rYt_s  =  a(s -r)  ,  £u^  =  ,  £u^.us  =  0  ,  t^s  ,  and  so  the 

expected  value  of  (8.19)  satisfies  the  following  equations: 


P 

(8.20)  l  6  a(s-r)  =  a2  ,  s  =0 

r=0  r 


P 

(8.21)  l  B  a(s-r)  =  0  ,  s  =  l,2,...,  . 

r=0  r 


The  above  equations  are  known  as  the  Yule-Walker  equations;  these  will 
be  discussed  further  in  Section  8.4. 

From  (8.21)  we  observe  that  the  sequence  cr(l-p),  o(2-p),..., 
ct(0),  ct(1),...  satisfies  a  homogeneous  difference  equation,  which  is 

the  same  as  the  homogeneous  difference  equation  (8.12).  Thus,  if  , 

P 

...,x  ,  the  roots  of  the  polynomial  equation  £  3  xp~r  =  0  ,  are 

p  r=0  r 

distinct  and  Bp  t  0  ,  the  solution  to  (8.21)  is  of  the  form 


P 

(8.22)  a(h)  =  l  c-xj,  h  =  1-p,  2-p,.  ..,0,1,..., 

i=l  1  1 


where  c^,...,c  are  coefficients. 
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There  are  p-1  boundary  conditions  of  the  form 

a(h)  =  a( -h )  ,  h  =  l,...,p-l  , 

and  the  other  boundary  condition  is  given  by  (8.20)  with  a(-p) 
replaced  by  a(p)  . 

Thus  the  behavior  of  the  autocovariance  function  of  an  AR(p) 
process  is  determined  by  the  general'  nature  of  (8.22).  We  study  this 
by  considering  some  special  cases. 

8.3.1  Special  Cases  Describing  the  Behavior  of  the  Autocovariance 
Function  of  an  Autoregressive  Process 

Following  Section  8.2.1  ,  we  consider  here  an  autoregressive  pro¬ 
cess  of  order  1  and  an  autoregressive  process  of  order  2. 

An  Autoregressive  Process  of  Order  1 

Suppose  that  in  (8.1)  p  =  l  (and  y=0),  so  that 

yt  +  ^1yt_1  =  ut>  t  =  2,3 .  . 

The  associated  polynomial  equation  x  +  =  0  has  one  root  . 

The  general  solution  is  (from  (8.22))  a(h)  =  Cj(-3j)^,  h=0,l,...  . 

From  (8.20)  we  have 

a2  =  g(0)  +  61<t(1)  =  c2  [1  +  61(-B1)  3  =  Cj  [  1  -  3j]  . 
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Hence  =  a2/( 1-g2)  ,  so  that 

a(h)  =  (-B1)ha2/(l-e^)  ,  h=0,l,...  . 

From  p(h)  =  a(h)/a(0)  ,  the  autocorrelation  function  is 

(8.23)  P(h)  =  (-01)h  ,  h  =  0,1 . 

If  |$.  |  <1  ,  then  we  have  the  important  useful  result  that  the 
theoretical  autocorrelation  function  of  an  autoregressive  process  of 
order  1  decays  exponentially  in  the  lag  h  .  The  decay  is  smooth  if 
3^  <  0  ,  and  it  alternates  in  sign  if  0^  >  0  .  In  Figure  8.3,  we  illus¬ 
trate  this  behavior  of  p(h)  for  nonnegative  values  of  h  .  We  also 
remark  that  the  behavior  of  p(h)  is  analogous  to  the  behavior  of  the 
weights  6^  discussed  in  Section  8.2.1  -  see  (8.14)  and  Figure  8.2. 

An  Autoregressive  Process  of  Order  2 

Now  suppose  that  in  (8.1)  p  =  2  (and  \i  =  0)  ,  so  that 

yt  +  elyt-l  +  S2yt-2  ut  ’  t  =  3’4’"”  • 

2  0 

The  associated  polynomial  equation  x  +  0^x  +  3,,x  =  0  has  the  roots 

xi  =  [-B1  ±A\  -4b2]/2  ,  i  =1,2  . 

If  the  roots  x^  and  x^  are  distinct,  then  a(h)  =  c^Xj  +  £2*2  » 
h  =  -1,0,1,...  .  Then  (8.20)  and  a(l)=o(-l)  can  be  solved  for  c^ 


and  C£  ,  yielding 


-C 


(M)§  uoj}D|ajjooo;ny 


Figure  8.3.  Behavior  of  the  theoretical  autocorrelation  function 
of  an  autoregressive  process  of  order  1. 
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h+1 


h+1 


(8.24)  a{ h)  = 


{■ 


^1 


(x^-XgHl-x^)  1-Xj  l-x2 


-) 


h  =  0,1,. 


If  we  require  that  | x^  [  <1  ,  i=l,2,  then  ^  and  e2  must  lie 
in  the  triangular  region  described  by  Figure  8.1;  that  is,  they  must 
satisfy  the  inequalities  (8.17).  Furthermore,  if  x^  and  x2  are 
real,  that  is  0^  and  e2  do  not  lie  in  the  parabolic  region  of 
Figure  8.1,  so  that  0^>432  ,  then  by  (8.24)  we  have  the  result  that 
a(h)  is  a  linear  combination  of  two  exponentially  decaying  functions 
of  h  ,  xxh+1  and  x2^+1  .  Depending  on  whether  the  dominant  root  is 
positive  or  negative,  a(h)  will  remain  positive  or  alternate  in  sign 
as  it  damps  out.  This  behavior  of  a(h)  as  a  function  of  h>0  ,  is 
shown  in  Figure  8.4. 

When  | x^  j  <1  ,  i  =1,2,  and  x^  and  x2  are  complex,  that 

2  i  e 

is,  <  4b2  ,  then  x^  and  x2  can  be  written  as  x^  =  ae  and 

^  i  0 

x2  =  ae"  ,  where  a<l  ,  and  now  (8.24)  becomes 


(8.25)  o(h)-°2°h^,n  e(htl>-°2  s™  , 

(1-a  ;sin  e[l-2a^  cos  20-hx  ] 


o' a  cos(eh-ij>) 


(l-a^)sin  e/l-2a^  cos  20+a^ 


h  =  0,1, . . . , 


p  2 

where  tan  <t>  =(l-a  )cot  e/(l+a  )  . 
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Thus  a(h)  is  a  damped  cosine  function  of  h  ;  the  behavior  of 
a(h)  as  a  function  of  h  =  0,  ±1,  ±2,...,  is  illustrated  in  Figure  8.5. 

Since  c(h)  is  a  linear  combination  of  the  hth  powers  of  the 
roots  and  »  both  which  are  less  than  1  in  absolute  value, 

| cr ( h )  |  is  bounded.  We  remark  that  the  behavior  of  a(h)  as  a  function 

of  h  is  analogous  to  the  behavior  of  the  weights  6^  as  a  function 
of  r,  discussed  in  Section  8.2.1  -  see  (8.16),  (8.18),  and  Figure 

8.2. 

Thus  to  conclude,  we  have  the  important  practical  result,  that 
when  and  0£  >  the  parameters  of  an  AR(2)  process,  lie  in  the 
triangular  region  described  by  Figure  8.1,  the  theoretical  autocorrela¬ 
tion  function  decays  either  exponentially  or  sinusoidally.  The  exponen¬ 
tial  decay  could  be  either  smooth  or  alternating  in  sign,  depending  on 
the  values  that  6^  and  take. 

In  Section  8.11,  we  show  the  behavior  of  the  estimated  autocorre¬ 
lation  function  of  some  real  life  data  which  we  claim  can  be  reasonably 
well  described  by  autoregressive  processes.  However,  in  order  to  be 
able  to  use  the  behavior  of  the  autocorrelation  function  as  a  means  of 
identifying  autoregressive  processes,  we  need  to  have  some  idea  about 
the  behavior  of  the  estimated  autocorrelation  function  of  some  known 
autoregressive  processes.  This  we  do  next,  and  also  make  some  other 
comments  which  have  some  practical  implications. 


Autocovanance 

a(h) 


40 


function  of  an  autoregressive  process  of  order 
when  its  associated  polynomial  equation  has 
complex  roots. 
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8.3.2  Behavior  of  the  Estimated  Autocorrelation  Function  of 
Some  Simulated  Autoregressive  Processes 

The  results  of  Section  8.3.1  can  be  generalized  in  a  straightfor¬ 
ward  manner  to  show  that  the  autocorrelation  function  of  autoregressive 
processes  must  decay  exponentially  or  sinusoidally.  Even  though  this 
result  is  true  in  theory,  it  is  unreasonable  to  expect  such  a  behavior 

of  the  estimated  autocorrelation  function.  Such  a  lack  of  conformance 
between  the  theory  and  its  application  is  mainly  due  to  the  sampling 

variability  in  our  estimate  of  the  autocorrelation  function  (see  Section 
7),  and  is  particularly  acute  when  we  are  dealing  with  series  of  short 
lengths,  wherein  our  estimate  of  the  autocorrelation  function  is  based 
on  few  observations.  Thus  a  good  deal  of  caution  and  insight  has  to  be 
used  in  order  to  identify  the  nature  of  an  underlying  stochastic  process 
by  examining  the  behavior  of  its  estimated  autocorrel ation  function. 

In  Table  8.1  we  give  r(h)  ,  the  values  of  the  estimated  autocorre¬ 
lation  function,  h = 0,1 , . . .  ,25,  based  on  250  computer  generated  obser¬ 
vations  from  an  AR(1)  process 

yt  -  •5y1._2  =  ,  t  =  2,3, . . .  ,250, 

with  y^  =  u^  .  A  plot  of  r(h)  versus  h  is  given  in  Figure  8.6. 
Barring  the  slight  aberrations  at  h=7,  8,  9,  13,  19,  and  23  ,  this 
plot  reveals  the  exponential  decay  pattern  expected  of  an  AR(1)  process 
with  3^  <  0  ,  and  | <  1  . 

In  Table  8.2  we  give  r(h)  ,  the  values  of  the  estimated  autocorre¬ 
lation  function,  h = 0,1, . . . ,25,  based  on  250  computer  generated  obser¬ 
vations  from  an  AR(2)  process 


Values  of  the  estimated  autocorrelation  function  r 
,1,...,25,  based  on  250  computer  generated  observa 


Estimated  Autocorrelation  r(h) 
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Figure  8.6.  A  plot  of  the  estimated  autocorrelation  function 
r(h)  versus  h  ,  h  = 0,1 , . . . ,25,  based  on  250 
computer  generated  observations  from  an  AR(1) 
process  with  3^  = - .5  . 


Values  of  the  estiriated  autocorrelation  function  r 
.1.....25.  based  on  250  computer  generated  observa- 
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yt  -  •9yt_1  +  -4yt_2  =ut  »  t  =  3,4,. ..,250, 

with  y2  =  .9y1+u2  and  y^  =  u^  .  A  plot  of  r(h)  versus  h  is  given 
in  Figure  8.7.  Since  82=-.9  and  =  *4  »  <  482  >  and  so  the 

roots  of  the  associated  polynomial  equation  are  complex,  (a  =  /A  =  .63, 
Xj,  =  .45  ±  .44i,  0  0,45°  )  Thus  the  theoretical  autocorrelation 

function  must  decay  sinusoidally;  this  feature  is  also  revealed  by  the 
estimated  autocorrelation  function  shown  in  Figure  8.7. 

In  Table  8.3  we  give  r(h)  ,  the  values  of  the  estimated  autocorre¬ 
lation  function,  for  h  = 0,1,. . . ,25,  based  on  250  computer  generated 
observations  from  an  AR(2)  process 

yt  +  •  5y j  ~  . 2y t_2  —  ,  t  —  3,4,...,  . 

A  plot  of  r(h)  versus  h  is  given  in  Figure  8.8.  Since  8j  =  .5, 
2 

and  82  = --2  ,  8^  >  482  ’  anc*  *ience  the  roots  the  associated  polynom¬ 
ial  equation  are  real.  These  roots  being  (-.5  ±  /.25+.8)/2  ,  it  is  clear 

-  5  -  1  025 

that  the  dominant  root  is  negative,  its  value  is  — - ^ - =  -*763  . 

Thus  according  to  the  material  in  Section  8.3.1,  the  autocorrel ation 
function  must  decay,  and  alternate  in  sign  as  it  does  so  -  see  Figure 
8.4.  The  estimated  autocorrelation  function  of  Figure  8.8  reveals  this 
tendency,  at  least  in  the  earlier  stages,  up  to  lag  10  or  so.  Later  on, 
the  estimated  autocorrelation  function  does  alternate  in  sign,  but  does 
not  decay.  We  attribute  our  reasons  for  this  to  the  sampling  variability 
of  the  estimates  of  the  autocorrelation  at  the  various  lags. 


Estimated  Autocorrelation  r(h) 
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Figure  8.7.  A  plot  of  the  estimated  autocorrelation  function 
r(h)  versus  h,  h  =  0,1,. . .  ,25,  based  on  250 
computer  generated  obervations  from  an  AR(2) 
process  with  =  -.9  and  =  *4  . 


Values  of  the  estimated  autocorrelation  function  r(h)  , 
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Figure  8.8.  A  plot  of  the  estimated  autocorrelation 
r(h)  versus  h,  h  = 0,1 , . . . ,25,  based 
computer  generated  observations  from  an 
process,  with  e,  =  .5  and  $p  =  -.Z  . 
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The  behavior  of  the  estimated  autocorrelation  function  of  some  real 
life  data  which  we  feel  can  be  reasonably  well  described  by  autoregressive 
processes  is  shown  at  the  end  of  this  section,  in  8.11. 

8.4  Expressing  the  Parameters  of  an  Autoregressive  Process  in 
Terms  of  its  Autocorrelations 

The  Yule-Walker  equations  (8.21)  enable  us  to  express  the  autoregres¬ 
sive  parameters  B,,...,3  in  terms  of  the  autocorrelations  p(s), 

^  r 

s  =  1 , . . . ,  .  To  see  this,  we  set  s=l,...,p  in  (8.21),  divide 
throughout  by  a(0)  ,  and  observe  that 

p(l)  =  -31  -  62°( 1)  “  3pp(p-l)  , 

p(2)  =  -31p(1)  -  32  -  ...  -  3pP(p-2)  , 

•  • 

•  • 

•  • 

p(p)  =  -31p(p-l)-62P(p-2)  -  ...  -  Bp  . 

(3  —  [3^9  •  •  •  s3p3  5  p  —  fp(l) » ••  •  sPvP)  ]  9  and 


1 

p(l) 

p(P-1) 

p(l) 

1 

p(P-2) 

• 

p(p-l) 

p(P-2) 

• 

1 

then  p  =  -  PB  from  which  we  have  (since  Pis  positive  definite) 
(8.27)  B  =  -  P_1p  . 


(8.26) 


If  we  denote 


The  matrix  P  is  unknown  as  the  autocorrelation  matrix. 
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Thus,  the  p  autoregressive  parameters  can  be  expressed  in  terms 

of  the  p  autocorrelations  p(l) , . . . , p(p)  .  This  feature  can  be  used 

to  estimate  s  ,  using  an  estimate  of  p  . 

2 

We  obtain  a  ,  the  variance  of  the  disturbance,  by  setting 
a(-r)=a(r)  in  the  Yule-Walker  equation  (8.20)  to  obtain 

(8.28)  60a(0)  +  8^(1)  +  ...  +  3pa(p)  =  a2  . 


8.5  The  Partial  Autocorrelation  Function  of  an  Autoregressive 
Process 

In  Section  8.3  we  have  shown  that  a(h)  ,  the  autocovariance  func¬ 
tion  of  an  autoregressive  process  of  order  p,  is  infinite  in  extent. 

Thus  from  (a(h)}  it  is  hard  to  determine  the  order  of  an  autoregressive 
process.  The  partial  autocorrelation  function,  to  be  discussed  here, 
will  help  us  in  determing  the  order  of  an  autoregressive  process. 

To  be  specific,  let  us  consider  a  stationary  autoregressive  process 
of  order  p 

yt  =  ut  '  6lyt-l  ~  ”•  "6pyt-p  ’  t  =  p+1,  p+2,... 

Recall  that  in  order  to  predict  y  We  need  consider  only  the  p 
lagged  variables  y^  ^,...,y^._  ,  since  the  other  variables  yt_p_^, 

yt-p-2’’”  ^ave  n0  effect  on  yt  • 
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The  partial  autocorrel ation  between  y^  and  y^_p  ,  to  be  denoted 
by  ir(p)  is  the  correlation  between  y^  and  y^  when  the  interme¬ 
diate  p-1  variables  yt-2’"*  ,yt-p+l  are  fixec1*"  That 

is,  tt(p)  is  the  correlation  between  y^  and  y^_p  when  the  interme¬ 
diate  variables  are  not  allowed  to  vary  and  exert  their  influence  on  the 
relationship  between  y  and  y.  .  Clearly,  ir(l)  ,  the  partial 

L  v"  U 

autocorrelation  between  y^  and  y^_^  ,  is  p(l)  ,  the  (ordinary)  auto¬ 
correlation  between  y^  and  y^  ^  ,  whereas  tt (0 )  the  partial  autocor¬ 
relation  between  y^  and  itself  is  1. 

Thus,  by  its  very  nature,  since  y.  , ,  have  no 

effect  on  y^  ,  the  partial  autocorrelation  function  of  an  autoregressive 

process  of  order  p  ,  ir(j)  f  0  ,  for  j  =0,1 . p  ,  and  ir(j)  =0  ,  for 

j  >p  .  The  fact  tt(  j )  vanished  for  j  >  p+1  ,  can  be  used  to  identify 

the  order  p  of  an  autoregressive  process,  provided  that  ir(j)  can  be 
computed . 

In  our  discussion  of  the  partial  autocorrelation  function  tt(p)  we 
had  mentioned  the  fact  that  the  intermediate  values  ^t-l * •• *’^t-p+l 
had  to  be  "held  fixed".  In  order  to  formalize  this  notion  we  shall  use 
some  results  which  are  standard  in  multivariate  analysis. 

Let  Y  =  [y^,  y^ , .. .  ,yt  ]  denote  the  vector  of  p+1  observa¬ 
tions,  and  let  z  denote  the  variance-covariance  matrix  of  these  p+1 
observations.  Suppose  that  Y  has  a  multivariate  normal  distribution 
with  mean  vector  0  and  covariance  matrix  z  ,  where 
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a(0) 

a(l) 

o(p) 

o(l) 

a(0) 

a(p-l) 

• 

• 

o(p) 

o(p-l) 

...  a(0) 

Let  us  rearrange  the  elements  of  Y  ,  and  partition  it  into  two 
component  sub-vectors  =  [y^,  y^_  ]  and  Y^  =  [yt_p  y^,..., 

yt.  +i]  .  Let  E22»  and  £-^2  be  the  variance-covariance  matrices 

of  Y^,  and  Y^  and  Y^  respectively.  That  is,  1^,  ^22^ 

and  Zj2  1S  a  partition  of  the  rearrangement  of  z  . 

(?)  ~  (2) 

Let  yv  '  be  a  particular  value  taken  by  the  vector  Yv  .  Then, 

it  can  be  shown  [Anderson  (1984),  p. 28 ]  that  the  conditional  distribu¬ 
tion  of  Y^  given  y^  is  a  multivariate  normal  with  mean  Ei2z22^^’ 

_i  1 

and  covariance  matrix  Ep  -  s  1  £E 2 2s  1 2  =  zll*2  *  say‘  Thl"s  1S  a  9eneraii~ 
zation  of  the  results  mentioned  in  Section  6. 

-1  (2) 

The  vector  ^-^22  *  is  called  the  regression  function  of  the 

regression  of  Y^  on  y^  .  The  matrix  z^^  is  a  2x2  matrix 
whose  elements  are  indicated  below: 
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Ttt*(t-1) ,. . . ,(t-p+l) 


Jt(t-p)*(t-l),...,(t-p+l) 


f(t-p)t*(t-l) .,(t-p+l)  a(t-p)(t-p)-(t-l) . , ( t-p+1) 


The  partial  correlation  between  y  and  y 

t  r-p 

(t-p+l)  fixed  at  y^  is 


holding  (t-1),.... 
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=  _  CTt(t-p)«(t-l) . (t-p+1) _  . 

/°tt*  (t- 1) ...  (t-p+1)  / a(t-p)  (t-p)  •  (t-1) , . . . ,  (t-p+1) 

note  that  ir(p)  is  independent  of  yv  . 

As  an  example,  if  Y  =  (yt,  yt_r  yt_2)\  and  if  Y^=(yt,  yt_2)' 

(2) 

and  Yv  '  =  y  ^  ,  then  the  partial  correlation  between  y^  and  y^._2  ’ 
tt(  2)  ,  turns  out  to  be 

"(2)  -  (p(2)  -  p2(l))/(l  - p2( 1) >  . 


8.5.1  Relationship  between  Partial  Autocorrelation  and  the 
Last  Coefficient  of  an  Autoregressive  Process 

An  interesting  relationship  between  -n-(p)  ,  the  partial  autocorrela¬ 
tion  of  y^  and  y^_p  ,  and  ^  ,  the  last  coefficient  of  an  autoregres¬ 
sive  process  of  order  p,  can  be  observed.  This  relationship  simplifies 
our  calculation  of  tt ( p)  ,  since  3  can  be  easily  obtained  from  the 

r 

Yule-Walker  equations  via  equation  (8.28). 

In  order  to  see  a  relationship  between  ir(p)  and  3^  ,  consider  an 
AR(2)  process 

yt  =  ut  '  3lyt-l  '  B2yt-2 


and  solve  the  resulting  Yule-Walker  equations  to  obtain 
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1 

p(l) 


p(1) 

p(2) 


1 

p(1) 


P(l) 

1 


>(2)  V(l) 

1  -  P2(l) 


2  2 

However  [  (p(  2)  -  p  (l))/(l-p  (1))]  is  indeed  the  partial  autocorrela¬ 
tion  between  and  ;  thus  tt(2)  =  -62  *  *n  a  sim^ar  manner, 

if  we  consider  an  AR(3)  process 

h  ‘  ut  -  elyt-l  '  tyt-2  -  63yt-3 


and  solve  the  resulting  Yule-Walker  equations,  we  observe  that 


6 


3 


1  p(1)  P ( 1) 

p(1)  1  P (2) 

P (2)  p(l)  P ( 3 ) 

1  p(1)  P (2) 

p(D  1  p(1) 

p(2)  p(1)  1 


which  again  can  be  verified  as  the  negative  of  the  partial  autocorrela¬ 
tion  between  y^  and  y^_^  . 

In  general,  we  observe  [Anderson  (1971),  pages  188  and  222]  that 
for  an  autoregressive  process  or  order  p  ,  -n- ( p)  the  partial  autocorre¬ 
lation  between  y^  and  y^  p  is  -g  ,  where 
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(8.29) 


1  p(l) 

p(2) 

...  p( 1) 

p(l)  1 

•  • 

p(l) 

• 

...  p(2) 

• 

•  • 

•  « 

p(p-l)  P ( p-2) 

• 

• 

p(p-3) 

• 

...  p(p) 

1 

P(D 

P  ( 2 ) 

p(p-l) 

p(1) 

• 

1 

p  ( 1 ) 

• 

p(p-2) 

• 

• 

• 

p(p-l) 

• 

• 

p(p-2) 

• 

• 

p(p-3) 

• 

• 

1 

It  is  helpful  to  remark  that  the  determinant  in  the  denominator  is 
simply  the  determinant  of  the  autocorrelation  matrix  for  an  AR(p) 
process  P  (Section  8.4),  whereas  the  matrix  in  the  numerator  is  P 
with  the  last  column  replaced  by  p(l) , . . . ,p(p)  . 

An  expression  for  ir(j)  the  partial  autocorrelation  between  yt 
and  y.  .  ,  can  be  obtained  if  we  write  the  Yule-Walker  equations  for  j  , 

J 

and  set  ir(j)  =  8 .  ,  where  3.  is  given  by  equation  (8.29);  recall  that 

vJ 

tt(0)  =1  ,  and  that  tt(1)  =p(1)  . 

The  partial  autocorrelation  function  is  a  plot  of  ir(h)  versus  h  , 
h  =  l,2,...,  ;  the  partial  autocorrelation  function  is  abbreviated  as 
PACF. 

We  estimate  ir(l)  by  r(l)  ,  and  estimate  ir(j)  by  tt(j)  ,  where 
tt ( j )  is  obtained  by  replacing  the  p(*)'s  in  (8.29)  by  their  estimates 
r(*)'s  . 
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8.5.2  Behavior  of  the  Estimated  Partial  Autocorrelation  Function 
of  Some  Simulated  Autoregressive  Processes 

Even  though  the  partial  autocorrelation  function  of  an  autoregres¬ 
sive  process  of  order  p  must  theoretically  vanish  at  lags  p+1,  P+2, 
...,  it  is  unreasonable  to  expect  such  a  behavior  of  the  estimated 
partial  autocorrelation  function.  The  reasons  for  this  are  analogous 
to  those  given  for  the  behavior  of  the  estimated  autocorrelation  func¬ 
tion  -  see  Section  8.3.2.  Thus  caution  and  insight  must  be  used  when 
identifying  the  order  of  an  autoregressive  process  by  examining  its 
estimated  partial  autocorrelation  function. 

In  Table  8.4  we  give  tt ( h )  ,  the  values  of  the  partial  autocorre¬ 
lation  function  for  h  =  0,l,...,25  based  on  250  computer  generated 
observations  from  the  AR(1)  process 

yt  “  ,5yt-l  =  ut  ’  t  =  2,3 . 

discussed  in  Seciton  8.3.2. 

A  plot  of  Tt(h)  versus  h  is  given  in  Figure  8.9.  Barring  some 
slight  aberrations  at  a  few  lags,  this  plot  reveals  the  behavior  that 
we  expect  from  the  PACF  of  an  AR(l)  process,  namely  that  tt ( 1 )  must  be 
significantly  different  from  0  ,  and  that  tt ( j )  ,  must  be  close  to  0 
for  j  =  2,3, . . . ,  . 

An  examination  of  Figures  8.6  and  8.9  reveals  the  desired  result 
that  for  an  AR(1)  process,  the  autocorrelation  function  decays  expo¬ 
nentially,  and  that  the  partial  autocorrelation  function  vanishes  after 
lag  1. 


Values  of  the  estimated  partial  autocorrelation  function  tt( h ) , 
=0,1..... 25,  based  on  computer  generated  observations  from  an 
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In  Table  8.5  we  give  ir(h)  ,  the  values  of  the  estimated  partial 
autocorrelation  function,  for  h = 0,1,. .  .,25,  based  on  250  computer 
generated  observations  from  the  AR(2)  process 

yt  -  •9yt_1  +  .4yt_2  =  ut  ,  t  =  3,4, . 

discussed  in  Section  8.3.2. 

A  plot  of  tt ( h )  versus  h  is  given  in  Figure  8.10.  As  is  to  be 
expected,  barring  some  minor  aberrations,  tt(1)  and  tt( 2 )  are  signi¬ 
ficantly  different  from  0,  and  -rr ( j )  is  close  to  zero,  for  j=3,4,...,. 

In  Table  8.6  we  give  rr(h)  ,  the  values  of  the  estimated  partial 
autocorrelation  function,  for  h = 0,1 , . . . ,25,  based  on  250  computer 
generated  observations  from  the  AR(2)  process 

y^.'*'*5y^._2  “  *2yt_2  ~  ^  ’  t  =  3,4,...  , 

discussed  in  Section  8.3.2.  A  plot  of  tt( h )  versus  h  is  given  in 
Figure  8.11.  Once  again,  as  is  to  be  expected,  tt(1)  and  tt(2)  are 
significantly  different  from  0  ,  and  tt ( j )  is  close  to  zero,  for 
j— 3,4,...,  . 

An  inspection  of  Figures  8.7  and  8.10,  and  Figures  8.8  and  8.11, 
reveals  the  desired  result  that  the  autocorrelation  function  of  an 
AR(2)  process  decays  either  exponentially  or  sinusoidally,  whereas  the 
partial  autocorrelation  function  vanishes  after  lag  2. 

The  behavior  of  the  estimated  autocorrelation  function  and  the 

partial  autocorrelation  function  of  some  real  life  data  which  we  believe 

can  be  reasonably  well  approximated  by  autoregressive  processes  is  shown 
in  Section  8.11. 
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Figure  8.10.  A  plot  of  the  estimated  partial  autocorrelation 
function  Tr(h)  versus  h,  h  =  0,1, . . .  ,25, 
based  on  250  computer  generated  observations 
from  an  AR(2)  process  with  6^  =  -.9  and 


Values  of  the  estimated  partial  autocorrelation  function  tt( h)  , 


O 

II 
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Figure  8.11.  A  plot  of  the  estimated  partial  autocorrelation  function 

£(h)  versus  h,  h  =  0,l . 25,  based  on  250  computer 

generated  observations  from  an  AR(2)  process  with 
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8.6  An  Explanation  of  the  Fluctuations  in  Autoregressive 
Processes 

A  typical  time  series  described  by  an  autoregressive  process  fluc¬ 
tuates  up  and  down  with  oscillations  which  are  not  regular,  but  whose 
average  length  depends  on  the  nature  of  the  underlying  difference  equa¬ 
tion.  We  can  offer  an  explanation  of  these  fluctuations  by  considering 
the  representation 

oo 

yt  =  I  Vt-r 
L  r=0 


and  noting  that 


^s+q  (S0Us+q  +  6lUs+q-l  + 


6  u  +  6  .  u  1  +  .. . 

q  s  q+1  s— 1 


Thus,  a  given  ug  will  influence  a  subsequent  ys+^  via  the  coeffici¬ 
ent  <5  .  In  Section  8.2  we  have  pointed  out  the  circumstances  under 
q 

which  the  coefficients  5^  oscillate,  causing  fluctuations  of  the 
successive  y^'s  .  To  illustrate  this  point  we  show  in  Figure  8.12,  a 
plot  of  a  time  series  comprising  of  100  observations  generated  on  a 
computer  by  an  AR(2)  process 


yt  " ’9yt-l  +  *4yt-2  =  ut  ’  t  =  3,4, .. . ,  . 


This  is  the  process  considered  in  Sections  8.3.2,  and  8.5.2.  Since 

2 

Bj(=-.9)  and  32 ( = - 4 )  are  such  that  6^  <  »  the  roots  of  the 

associated  polynomial  equation  are  complex  with  0  =44.68°,  and  a  =  .634 
(Section  8.2.1).  Thus  the  coefficients  6r  will  have  a  damped 
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sinusoidal  behavior  of  the  type  indicated  in  Figure  8.2.  Substituting 
the  above  values  of  e  and  a  in  (8.18),  we  observe  that  6^ =  .899, 

<$2  =  .409 ,  63  =  .008,  64  =  -  .156,  <$5  =  - .  144 ,  6g=-.06,  67  =  -. 003,  <5g  =  .025, 
6g  =  .023,  and  =  .011  ;  the  remaining  values  of  <5r  ,  for  r>l  are 
all  less  than  .002  and  are  thus  essentially  0  .  It  is  because  of  the 
above  behavior  of  the  <5r's  that  the  observations  yt  fluctuate  up  and 
down  about  0  with  an  average  length  of  oscillation  of  about  length 
10  -  see  Figure  8.12.  It  is  also  useful  to  keep  in  mind  Figure  8.7,  the 
estimated  autocorrelation  function  of  the  generated  series,  and  note 
that  the  estimated  autocorrelations  for  lags  greather  than  10  are,  barr¬ 
ing  sampling  variability,  essentially  small. 

8.7  Autoregressive  Processes  with  Independent  Variables 

Suppose  that  there  are  m  independent  variables  zlt . 

that  are  known  to  affect  the  time  series  {y^}  ,  t  =  l,2,...,  being 
investigated.  The  effect  of  these  m  variables  can  be  incorporated 
into  an  AR(p)  model,  by  writing 

P  m 

(8.30)  l  f3V  +  l  y.z.t  =  u  ,  t  =  p+l . 

r=0  r  i=l  1  'z  z 

where  Yj,...,y  are  constants.  It  is  of  interest  to  compare  the 

model  (8.30)  with  the  classicial  regression  model  (2.1)  of  Part  I. 

Let  x.,...,x  be  the  roots  of  the  associated  polynomial  equation 
P  p 

of  the  stochastic  difference  equation  [  ?y,  =  u.  ,  t  =  p+1 . 

r=0  r  ^ 


Values 
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or  of  100  computer  generated  observations 
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Then,  using  the  forward  lag  operator  p,  where  pyt=yt+$ 
P 

l  3_y+  „  can  also  be  written  as 
r=0  ^ t_r 


(8.31) 


l  3ryt  =  l  e„pp"r  yf  _  =  n  (p-x-)yt  n 
=0  r  1  r  r=0  r  r'p  i=l  1  t_p 


In  terms  of  the  operator  £  ,  the  above  becomes 


(  I  e-xs)yx  =  n  (p-x.)yt 

s=0  s  r  i=l  1 


P  * 


or 


(  l  3  xs)_1yt  ■  (  n  (p-x.))"^  . 
s=0  s  r  p  i=l  1  1 


Thus,  for  t  =  p+l,  p+2,...,  the  model  (8.30)  becomes 


p  m 

ut =  .^(^Xj)yt-p +  Tj  Y1ZU 

p  m  p  , 

=  n  (p-xj  [y.  +  l  y,(  n  (p-x.))  zitl 

j=l  J  i=l  1  j=l  J  ir 


m 


n  (p-x.)[yt  +  l  Yi(  I  3  Xs)"1  z.  ] 

j=l  J  r  p  i=l  1  s=0  s  p 


Before  proceeding  further,  it  is  helpful  to  recall  a  result  that 
we  have  encountered  in  Section  8.1.1,  namely,  that 


( i  3  /r1  =  i  «/  , 

r=0  r  r=0  r 


where  the  6^'s  are  the  coefficients  in  the  equality 
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(  l  e  zr)_1  =  l  s  zr  ; 
r= 0  r  r=0  r 


see  (8.8).  Using  the  above,  we  now  write  as 


or 


m 


u.  =  n  (p-x.)[yt  +  l  Yi  l  6  £  z.  t  ] 

t  ,=1  j  z  p  i=1  1  s=0  s  btp 


m 


Cyt-P  +  i|1  5szi,t-p-s]  » 


m  » 


(8.32) 

Ut  =  , 

s  i  nee 

A  special 

case  i 

this,  suppose 

that  i 

becomes 

P 

(8.33) 

l 

r=0 

where  p  =  -Yj 

<o 

c 

8  cx  I 

ut  =  l  B  [yt  +  l  y1  l  Vi  t_r_sl 
t  r=0  r  z  r  i=l  1  s=0  s  1,1  r  s 


n  (p-x.)y.  =  l  3  v 

j=l  3  z  p  r=0  r  z  1 


'It 


t  =  p+1 ,  p+2,. . .  , 
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8.8  Stationary  Autoregressive  Processes  Whose  Associated 
Polynomial  Equations  Have  at  Least  One  Root  Equal  to  1 


Much  of  our  discussion  thus  far,  has  been  based  on  the  requirement 


that  al 1  the  roots  of  the  associated  polynomial  equation  £  3  xr  =  0 

p 

of  an  AR(p)  process  l  3  £ry+  =  u+  ,  t  =  p+l,  p+2,...,  be  less  than 

r=0  r  r  r 


1  in  absolute  value.  We  have  also  assumed  that  the  sequence  of  random 
variables  yi,yo,...  described  by  the  AR(p)  process  be  stationary. 
In  this  section,  we  investigate  the  implications  of  allowing  the  abso¬ 
lute  value  of  one  or  more  roots  of  the  associated  polynomial  equation 
take  a  value  equal  to  1  and  still  maintain  the  requirement  that  the 
underlying  sequence  of  random  variables  be  stationary. 

We  begin  by  considering  a  stationary  autoregressive  process  of 
order  1  with  its  single  root  taking  a  value  1;  thus  we  have 


yt=yt_2  +  ut  »  t  =  2,3,...  , 


or 

Ayt-l=ut’  t  =  2,3, . . .  . 

Thus  the  first  difference  of  our  autoregressive  process  of  order  1  with 

its  single  root  equal  to  1,  is  described  by  an  innovation  process.  This 

2  2 

latter  process  is  stationary.  We  let  eut=0,  and  eu^  =  a  for  all 
values  of  t  . 

Now  for  all  s  > 0  ,  we  note  that 

yt  “  yt-s  =  ut  +  ut-l  +  *  •  •  +  ut-s+l 
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co  that 

e(yt-yt-s)2  =  O'2  +  -  aa^t-s  *  sa?  • 

2  2 

Since  our  sequence  {y^}  is  stationary,  Sy^  =  Sy^_s  ,  and  so 

2  sa2 

Sy ^.y^._g  -  o  ( s )  =  gy ^  ^  »  s  =  l,2,...,. 

2 

The  above  result  can  hold  for  all  s>0  only  if  a  =0  ,  in 
which  case  y^  =  s  »  with  probability  1. 

To  generalize,  we  consider  a  stationary  autoregressi ve  process  of 
order  p  ,  p>l  ,  and  allow  one  root,  say  to  equal  1  ,  and  require 
that  the  other  p-1  roots  are  less  than  1  in  absolute  value;  that  is, 
|x. |  <  1  ,  i  =2,...,p  .  Following  (8.31),  we  may  write  our  stationary 

AR(p)  process  as 


P 

(p-l).n2  (p-x,.)yt_p  =  ut 


P 

If  we  let  _n  (p-x^)y^._p  =  ,  then  our  AR(p)  process  can  be 

written  as  (p-l)zt-1  =  u^  ,  or  since  (p-l)=A,  we  have  Azt_^=ut 
It  now  follows  from  our  previous  discussion  of  the  stationary 
AR(1)  process  with  a  single  root  equal  to  1  that  for  all  s>0  , 
zt  =  zt_s  =  z  ,  say.  Thus 


P 

n  (p-x.)y  =  z  , 

i=2  1  r  p 
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and  yt  =  e”_q  6$z  ,  so  that  yt=yt_$  >  with  probability  1  .  We 
therefore  have  as 

Theorem  8.3:  If  a  stationary  autoregressive  process  of  order  p  has 
at  least  one  root  of  its  associated  polynomial  equation  equal  to  1  , 
then  all  values  of  the  process  are  the  same  with  probability  1  . 


8.9  Some  Linear  Nonstationary  Processes 

We  shall  now  introduce  a  type  of  nonstationary  stochastic  processes 
that  are  suitable  for  describing  many  empirical  time  series.  Such  series 
behave  as  though  they  have  no  fixed  mean.  The  stochastic  processes 
introduced  here,  are  within  the  general  structure  of  autoregressive 
processes. 

Suppose  that  a  nonstationary  sequence  of  random  variables  y^y,,, 

...  is  described  by  a  stochastic  difference  equation  of  order  p+d  , 
so  that  (pp+d  +  e1PP+d“1  +  ...+  Sp+d  P°)yt_p_d  =  ut  »  t  =  p+d+1  , 
p+d+2,...,  or  equivalently,  the  sequence  is  described  by  an  autoregres¬ 
sive  process  of  order  p+d  ,  where 

p+d 

I;  s/y+  =  u+  ,  t  =  p+d+1,  p+d+2,... 

r=0  r  r  r 

P+d  D+d_r 

The  associated  polynomial  equation  ),  B  xK  =  0  of  the  above 

r=0  r 

process  has  p+d  roots  x^, . ..  ,Xp,x  +^,. . .  ,Xp+cj  .  Suppose  that  d  of 

these  roots,  say  xp+1, . . .  ,xp+d  are  exactly  equal  to  1  ,  and  that  the 
remaining  p ( >  1 )  roots  x,,...,x  ,  are  less  than  1  in  absolute  value. 

Then,  following  (8.31),  we  can  write  our  AR(p+d)  process  as 
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n  (p-x.)(s^i)dy 

i=l  1 


t-p-d 


u 


t 


or 


(8.34) 


n  (p-x.)  a  y 
i  =1  1 


t-p-d 


t  =  p+d+l,  p+d+2 . 


since  p-1  =  A  . 

If  we  let  w ^  ,  =  a'V  ,  ,  and  assume  that  the  differenced 

t-p-d  J t-p-d  ’ 

sequence  (wt  p_d>  ,  t  =  p+d+l,  p+d+2 .  is  stationary,  then  for 

p  >l  (8.34)  becomes 

p 

(8.35)  (p-xi)wt-p-d  =  ut  ,  t  =  p+d+1,  p+d+2, . 

Thus  for  p  > 1  our  model  for  the  nonstationary  sequence  {y^}  , 

t  =  l,2,...,  is  one  for  which  {wt_p_d>  ,  t  =  p+d+1  ,  p+d+2 .  its 

d-th  difference  is  described  by  a  stationary  autoregressive  process  of 
order  p  .  Since  the  roots  x^,...,x  ,  are  assumed  to  be  less  than 
1  in  absolute  value,  all  our  previous  results  for  stationary  autore¬ 
gressive  processes  are  also  applicable  for  the  model  (8.35). 

When  p  =  0  ,  and  d  =  l  ,  the  first  difference  of  our  nonstationary 

sequence  {y^}  is  described  by  an  innovation  process  (ut>  which  by 
definition  is  always  stationary;  see  Section  8.8. 

Note  that  if  the  original  series  (yt }  ,  t  =  l,2,...,  consists  of 
n  observations,  then  the  differenced  series  (w^. }  will  consist  of 

n-d  observations.  Since  wt_p_d  =  Adyt_p_d  ,  t  =  p+d+1,  p+d+2 . 

we  write  yt_p_d  =  A~dwt_p_ci  »  where  the  notation  A-d  needs  to  be 
explained.  For  this  purpose  we  set  d  =  l  ,  substitute  the  values 
t  =  p+2,  p+3 .  in  the  telescoped  series  wt  p  1=yt  p-yt  p  ^  ,  and 
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observe  that  we  can  write  y^_p  =  wt_p_^  +  w^-p-2  +  •••  +  wi  +  »  f°r 

any  t  >  p+2  .  The  operator  A-^  therefore  represents  summation  or 
integration  -  the  reverse  of  differencing  -  and  it  is  for  this  reason 
that  we  say  that  the  sequence  {y^}  ,  t  =1,2,...,  is  described  as  an 
integrated  autoregressive  process  of  order  p.  An  explanation  for 
A-d  ,  d  >  2  ,  follows  by  an  analogous  argument. 

8.9.1  Behavior  of  Estimated  Covariance  Functions  of  Integrated 
Autoregressive  Processes  and  Processes  with  an  Underlying 
Trend 

Since  the  associated  polynomial  equation  of  the  process  described 
by  the  difference  equation 

(8.36)  (pP+d  +  +  .. .  +  Sp+dP0)yt_p_d  =  ut  ,  t  =  p+d+1 . 

P+d  n+rl-r 

is  l  3  =  0  ,  it  follows  from  (8.22)  that  if  the  roots  x. , 

r=0  r  1 

...,Xp+d  are  distinct,  |x^ |  < 1  ,  j  =  1, . . . ,p+d,  and  2p+d  t  0  ,  o(h)  , 
the  covariance  between  y^  and  yt+^  ,  is 

p+d  , 

a(h)  =  l  c.xV  ,  h  =  1-p-d,  2-p-d . 0  , 

i=l  1  1 

where  Cj,...,c  d  are  coefficients. 

Since  the  roots  are  assumed  to  be  real,  distinct,  and  less  than  1 

in  absolute  value,  each  x!?  damps  exponentially.  If  a  pair  of  roots, 

say  x.  and  x,  are  conjugate  complex;  then  o(h)  is  a  mixture  of 
J  K 
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damped  exponentials  and  damped  sine  waves.  Now  suppose  that  one  of  the 
roots,  say  is  close  to  1  ,  so  that  for  some  small  number  6  >0  , 

Xjl  =  i-6  . 

Then  using  a  first  order  Taylor's  series  expansion  for  (l-s)n  ,  we  see 

U 

that  ( 1-6 ) n  ~  1  -  hs  ,  for  an  arbitrary  h  ,  and  so  c  x^  contributes 

36  36 

a  term  approximately  c  (l-h6)  to  a(h)  .  The  term  c  (l-hs)  decreases 

36  36 

linearly  and  slowly  in  h  ,  for  h  not  too  large.  Thus,  the  tendency  of 
the  estimated  autocorrelation  function  to  decrease  linearly  and  slowly  in 
h  ,  indicates  the  possible  presence  of  a  root  close  to  1  (in  absolute 
value),  in  (8.36),  the  associated  polynomial  equation  of  the  process. 

When  such  is  the  case  and  the  underlying  series  cannot  be  assumed 
stationary  (for  otherwise  the  result  of  Theorem  8.3  would  come  into 
effect),  we  may  want  to  consider  appropriate  differences  of  the  series 
and  attempt  to  model  these  as  a  stationary  autoregressive  process;  see 
the  discussion  following  (8.35). 

To  illustrate  the  above  issues,  we  generate  on  a  computer  250 
observations  from  an  AR(1)  process 

yt  -  =  ut  >  t  =  2,3, . . .  . 

A  time  series  plot  of  these  250  observations  is  shown  in  Figure  8.13. 

An  examination  of  this  plot  reveals  that  the  series  behaves  as  though 
it  has  no  fixed  mean.  This  is  to  be  expected  since  =-.99  being 
close  to  1  in  absolute  value  makes  the  nonstationarity  of  the  gener¬ 
ated  series  a  likely  possibility.  In  Table  8.7  we  give  values  of  r(h). 
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Fiqure  8.13.  A  plot  showing  the  behavior  of  250  computer  generated  observations  from  an  AR(1) 
process  yt  -  •99yt_1  =  . 


Values  of  the  estimated  autocorrelation  function  r(h)  , 

1 . 50,  based  on  250  computer  generated  observations  from 
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the  estimated  autocorrelation  function,  for  h = 0,1,. . . ,50,  and  in 
Figure  8.14  we  show  a  plot  of  r(h)  versus  h  .  As  is  to  be  expected, 
this  plot  conspicuously  shows  the  slow,  and  almost  linear  decay,  of  the 
estimated  autocorrelation  function. 


Since  the  estimated  autocorrelation  function  of  the  generated 
series  decays  slowly  and  linearly,  we  consider  wt  = ->^+1  " yt  ’  t  =  l,2, 
...,  the  first  difference  of  the  generated  series,  and  investigate  the 


behavior  of  its  estimated  autocorrelation  function.  Recall,  that  if 
were  to  be  exactly  equal  to  -1  ,  then  the  w^'s  would  be  described 

by  an  innovation  process  whose  autocorrelations  at  all  lags  other  than 


0  ,  is  zero. 


In  Figure  8.15,  we  show  a  plot  of  the  time  series  generated  by  the 


wt's  ,  for  t = 1,2, ...,249.  In  contrast  to  Figure  8.13,  we  see  that  the 
differenced  series  {w^}  reveals  fluctuations  around  a  fixed  mean  of 
zero.  In  Table  8.8  we  give  values  of  r(h)  ,  the  estimated  autocorrela¬ 
tion  function  of  the  w^  series,  for  h  =  0,l . 25,  and  in  Figure 

8.16  we  show  a  plot  of  r(h)  versus  h  .  We  contrast  this  plot  in 
Figure  8.16  with  that  of  Figure  8.14,  and  note  that  in  the  former,  as  is 
to  be  expected,  the  autocorrelations  at  lags  other  than  0  are,  barring 
sampling  variability,  effectively  zero. 


8.9.2  The  Covariance  Function  of  Some  Processes  with  an 


As  a  note  of  caution,  it  is  not  true  that  the  tendency  of  the 
estimated  autocorrelation  function  to  decrease  slowly  necessarily 
implies  that  a  root  close  to  1  exists.  Such  a  tendency  can  also  be 
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Figure  8.14.  A  plot  of  the  estimated  correlation  function  r(h)  versus 
h  =  0,l,...,50  based  on  250  computer  generated  observations 
from  an  AR(1)  process  with  e,  =  -.99  . 
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generated 


Values  of  the  estimated  autocorrelation  function  r(h)  , 


80 


m 

c: 

o 

•r* 

4- > 

03 

> 

5- 
<D 
CO 

-Q 

O 

“D 

O) 

4- > 

03 

5- 

<u 

c: 

(D 

cn 

S- 

O)  CD 

+->  cd 

23  • 

CL  I 

E 

O  II 
o 

! - 

o  CO. 

LO 

CM 

<D  4-> 
JC  *i- 

■M  S 

C|_  to 

O  to 
a; 

CO  U 


a; 

u 

c: 

<u 

S- 

<u  - 


o 

£- 

Q_ 


•i-  oc: 
■a  <C 

+-> 

to  c: 

S-  03 
■r* 

M-  E 
O 
O)  S- 


S- 

o 


r» 

LO 

CM 


O 

II 


CM 

pH 1 

CO 

o 

1 

25 

pH 

1 

pH 

pH 

CD 

O 

24 

90* 

1 

o 

pH 

.03 

23 

.03 

CD 

-.06 

22 

90* 

pH 

00 

pH 

* 

pH 

CM 

o 

• 

1 

I 

pH 

pH 

o 

• 

20 

o 

• 

1 

1 

CO 

03 

CD 

07 

• 

_ *1 

LO 

O 

pH 

• 

CO 

pH 

-.03 

.04 

pH 

.02 

1 

oo 

-.06 

LO 

pH 

pH 

o 

CD 

CO 

CM 

O 

• 

to 

pH 

o 

1 

i 

pH 

CM 

O 

^1" 

pH 

pH 

o 

• 

1 

LO 

CO 

o 

o 

pH 

• 

1 

-SZ 

-SZ 

_ 

S- 

i- 

<4- 

M— 

o 

O 

-C 

a; 

JZ 

<13 

23 

23 

CD 

r— 

CD 

p— 

03 

03 

03 

03 

_ 1 

>■ 

> 

81 


(q)j  uoijDiajjoDOjnv  paioujjjsg 


Fiqure  8.16.  A  plot  of  the  estimated  correlation  function  r(h)  versus 
h  ,  h  =0,1,.. .,25  of  the  first  differences  of  the  250 
computer  generated  observations  from  an  AR(1)  process 
with  B,  =  -.99  . 


-82- 


observed  whenever  there  is  an  underlying  trend  in  the  series.  To  see 
why  this  is  so,  let  us  consider,  for  example,  a  process  with  an  under¬ 
lying  linear  trend  of  the  form 

yt  =  ut  +  t  ,  t  =  1,2,...  , 

where,  as  before,  UpU^,...  are  independent  and  identically 

2 

distributed  with  mean  0  and  variance  a  . 

Since  ey^.  =  t  for  all  values  of  t  ,  it  is  easy  to  verify  that 
for  a  series  of  length  T  the  expected  value  of  the  sample  mean  of 
yr...,yT  is 


If  we  are  to  use  the  numerator  of  (7.4)  to  compute  the  covariance 
of  the  T  observations  at  lag  h>0,  then  the  theoretical  quantity 
that  is  being  estimated  is 

T  h 

(8*37)  £  T  (yt  "  "T^t+h  "  ^  * 

Using  the  fact  that  ey^y^  =  t(t+h)  ,  h  =  l,2,...,  and  that 
t3  =  (T-h) (T-h+1) (2T-2h+l)/6  ,  we  can  show  that  for  large  values 
of  T,  (8.37)  can  be  approximated  by  (T-h) ( (T-h)3  -  3h3)/24  ,  which 
for  small  values  of  h  decreases  slowly  in  h  . 

An  analogous  conclusion  can  be  drawn  for  other  types  of  processes 
and  other  types  of  trends.  Thus  in  practice,  to  investigate  the 
nature  of  the  series,  by  plotting  it  to  see  whether  it  exhibits  an 
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underlying  trend.  If  the  underlying  trend  appears  to  be  a  polynomial, 
then,  as  discussed  in  Section  3.3,  such  a  trend  can  be  eliminated  by 
taking  an  appropriate  number  of  differences  of  the  series. 

We  have  thus  seen  that  the  differencing  of  an  observed  series  may 
be  motivated  by  two  distinct  considerations.  The  first  enables  us  to 
model  certain  types  of  nonstationary  sequence  of  random  variables,  via 
the  mechanism  of  a  integrated  autoregressive  processes,  whereas  the 
second  enables  us  to  eliminate  the  presence  of  an  underlying  polynomial 
trend  in  a  series. 

To  illustrate  the  effects  of  a  linear  trend  on  the  estimated  auto¬ 
correlation  function  of  a  series,  we  generate  on  the  computer,  250 
observations  from  an  autoregressive  process  of  order  1  with  ^  =  -.5  , 
and  for  which  a  linear  trend  term  is  added.  Note  that  this  is  the 
same  series  considered  in  Sections  8.3.2  and  8.5.2,  except  that  the 
inclusion  of  a  linear  trend  term  makes  the  generated  series  nonstationary. 

In  Table  8.9  we  give  r(h)  ,  the  values  of  the  estimated  autocorre¬ 
lation  function,  for  h = 0,1,. .. ,25,  for  the  250  computer  generated 
observations  described  above.  In  Figure  8.17,  we  plot  r(h)  versus  h. 
This  plot  clearly  shows  the  very  slow  decay  of  the  estimated  autocorrela¬ 
tion  function.  The  estimated  autocorrelation  and  partial  autocorrelation 
functions  of  the  first  difference  of  this  series  will  reveal  a  behavior 
analogous  to  those  of  Figures  8.6  and  8.9,  since  by  differencing  the 
series  we  would  have  eliminated  the  linear  trend. 

An  example  of  some  real  life  data  with  a  trend,  and  for  which  the 
estimated  autocorrelation  function  decreases  linearly  and  slowly  is 
given  in  the  next  section. 


Values  of  the  estimated  autocorrelation  function  r(h)  , 


Estimated  Autocorrelation  r(h) 
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Figure  8.17 .  A  plot  of  the  estimated  autocorrelation  function 
r(h)  versus  h,  h  =  0,1 , . . .  ,25,  based  on  250 
computer  generated  observations  from  an  AR(1) 
process  with  6.  =-.5  ,  and  a  linear  trend  term 
added  to  it. 
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8.9.3  Behavior  of  the  Estimated  Autocorrel ation  Function  of  a 
Real  Life  Nonstationary  Time  Series 

In  Figure  8.18,  we  show  a  plot  of  6^  ,  the  US  gross  national 
product  (GNP)  in  billions  of  US  dollars,  for  the  years  t = 1920  through 
t = 1979  (Source:  Bureau  of  Economic  Analysis,  U.S.  Department  of 
Commerce,  Washington,  D.C.).  This  plot  indicates  that  the  GNP  series 
is  a  nonstationary  one,  it  being  increasing  (approximately)  exponentially 
over  time.  Thus  it  appears  reasonable  to  first  take  the  natural  loga¬ 
rithms  of  the  GNP,  lnG^  .  A  plot  of  y^ = 1 nG^ x 1000  versus  t  is 
shown  in  Figure  8.19;  the  actual  values  of  y^  are  given  in  Table  8.10. 
Figure  8.19  shows  that  the  yt  series  is  also  not  stationary,  it  being 
increasing  (approximately)  linearly  in  t  .  The  estimated  autocorrela¬ 
tion  function  of  the  y^  series,  for  lags  0  through  20  ,  is  shown  in 
Figure  8.20.  Because  this  estimated  autocorrelation  function  decreases 
linearly  and  slowly,  we  consider  the  first  differences  of  the  y^  series, 
w^  =  Ay^ = y - y ^  ,  t  =  1 ,2, . . . ,59  .  A  plot  of  versus  t  is  shown 
in  Figure  8.21;  the  actual  values  of  are  also  given  in  Table  8.10- 
From  Figure  8,21  we  see  that  whereas  the  w^  series  appears  to  have  a 
constant  level  (mean),  its  fluctuations  in  the  earlier  years,  1920-1947, 
appear  to  be  more  erratic  than  the  fluctuations  in  the  latter  years, 
1947-1979.  A  possible  explanation  for  this  behavior  is  that  "automatic 
stabilizers"  such  as  unemployment  insurance,  workmens  compensation,  etc., 
which  were  introduced  into  the  economy  as  of  1947,  tend  to  make  the  GNP 
less  erratic.  Plots  of  the  estimated  autocorrelation  and  partial  auto¬ 
correlation  functions  of  the  w^  series  are  shown  in  Figures  8.22  and 
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(The  quadratic  curve  superimposed  on  the  Gross  National  Product 
indicates  the  nature  of  the  trend.) 
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(The  straight  line  superimposed  on  the  logarithm  of  the  Gross 
National  Product  indicates  the  nature  of  the  trend.) 


Values  of  y.  ,  the  natural  logarithm  of  the  Gross  National  Product x  1000  , 
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Table  8.10  (continued) 
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Estimated  Autocorrelation  r(h) 
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Figure  8.20.  The  estimated  autocorrelation  function  of  yt  , 
the  Logarithm  of  GNP  times  1000. 
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Figure  6.21 .  A  plot  of  w  ,  the  first  difference  of  InGNPxlOOO  versus 
time,  for  1920  through  1979. 
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8.23,  respectively .  In  contrast  to  Figure  8.19,  we  note  from  Figure 
8.22  that  the  autocorrelation  function  at  lag  1  is  significantly 
different  from  0  and  takes  smaller  values  for  the  other  lags.  How¬ 
ever,  it  is  difficult  for  us  to  claim  in  Figure  8.22,  a  pattern  of 
either  an  exponential  or  a  sinusoidal  decay.  A  similar  type  difficulty 
is  apparent  in  Figure  8.23.  Thus  it  appears  that  the  wt  series 
cannot  be  reasonably  well  described  by  an  autoregressive  process  of  the 
type  discussed  here;  more  complicated  models  to  be  presented  later  on, 
may  be  necessary  for  analyzing  this  data.  Our  main  goal  here,  is  to 
show  Figure  8.20. 

8.10  Forecasting  (Prediction)  for  Stationary  Autoregressive 
Processes 

Suppose  that  a  sequence  of  random  variables  (yt>  ,  t  =  l,2,..., 
can  be  described  by  an  autoregressive  process  of  order  p 

yt  +  Blyt-l  +  B2yt-2  +  ”  ’  +  0pyt-p  =  ut  ’  t  =  p+1,  P+2>*-*>  • 

Let  yt-2’*"’  °bserved  values  of  the  random  variables 

yt-l’  yt-2’  ’ *  ‘  '  0ur  goal  in  this  sectl’on  is  to  determine  a 
procedure  y/hich  would  give  us  a  best,  in  the  sense  of  minimum  mean 

square  error,  forecast  (predictor)  of  the  unobserved  variable  yt  . 

We  shall  soon  see  that  if  the  roots  of  the  associated  polynomial  equa- 
P 

tion  £  3  xg"r  =  0  of  the  AR(p)  process  given  above  are  all  less 
r=0  r 

than  1  in  absolute  value,  then  the  best  predictor  of  yt  is  indeed 


Estimated  Partial  Autocorrelation  jr(h)  Estimated  Autocorrelation  r(h) 
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the  natural  quantity 

“6lyt-l  "  B2yt-2  "  •••  “  epyt-p  ' 

To  see  that  the  above  is  true,  we  first  note  that  under  the  condi¬ 
tions  of  Theorem  8.1  in  the  equation 

yt'"8lyt-l  "62yt-2  "  •••'  Vt-P  +  Ut 


Thus,  the  conditional  expec- 
★ 

is 


u^  is  independent  of  1’  yt  2s*'* 

k 

tation  of  yt  given  that  y^  =  yt_1 ,  yt_2  =  yt_2  » 

"k  "k  k  k  k 

e(yt  |y^_j  =yt.i>  yt-2~  yt-2s  *  *  *  *  ^  —  -^iyt-i  ~  ^2yt-2  ~  ~  ^pyt-p 


the  right-hand  side  of  the  above  equation  can  be  used  to  forecast  yt  , 

k  k 

given  yt_jt  yt_2»***»  * 

k  k 

Now  let  f  be  any  ot*ier  function  of  the  previous 

*  *  *  *  . 
values  yt_^>  yt-2’***s  anc*  suPPose  that  we  use  f(yt_i>  y 

k 

as  a  another  forecast  of  yt  .  Then  the  mean  square  error  of  f(yt_j» 
* 

yt-2s***  '  as  a  Predictor  °f  yt  is 

e[f^yt-i”  *  *^_yt^  =£[f(yt-is*”)+eiyt-i+  ••*  +Bpyt-p_ut'' 

-eut+e[f(yt-i’---)+6iyt-i+  • 

since  u^  is  independent  of  y^_^,  yt_2»*** 

k  k  k  k 

The  above  is  minimized  when  f(yt_^,  y^,. •  •)  =  _e]yt-l  ~  &2yt-2  " 

**•  "  3pyt-p  ' 
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In  general,  when  the  conditions  of  Theorem  8.1  are  satisifed,  the 
best  (minimum  mean  square  error)  forecast  of  y^  given  yt_s_^> 

yt-s-2.— •  (s-°)  is  £-(y 1 1 y t _ s — l *  yt-s-2’ * ^ ’  the  conditional 

expectation  of  yt  given  yt_s_15  yt_s_2»***»  where  £{ytl-yt-s-l ’ 

yt-s-2’ * ’ ’  }  =  aslyt-s-l  +  as2yt-s-2  +  •••  +  aSpyt-s-p  *  and  asl  = 

•jlf 

as^,...,as  =  asp  are  given  in  equation  (8.6). 


8.11  Examples  of  Some  Real  Life  Time  Series  Described  by 
Autoregressive  Processes 

In  this  section  our  aim  is  to  demonstrate  the  methods  of  the  pre¬ 
vious  sections  by  considering  some  real  life  data  which  can  be  reason¬ 
ably  well  described  by  autoregressive  processes.  It  is  entirely 
possible  that  the  data  can  also  be  described  by  some  of  the  other  models 
introduced  later  on.  However,  at  this  point  in  time,  it  is  convenient 
to  introduce  the  data  and  use  these  to  indicate  the  practical  usefulness 
of  autoregressive  processes  of  a  simple  order. 

8.11.1  The  Weekly  Rotary  Rigs  in  Use  Data* 

A  rotary  rig  is  composed  of  five  major  components  and  costs 
upward  of  $500,000  each.  It  is  used  for  drilling  for  oil  and  gas.  The 
number  of  rotary  rigs  in  use  per  week,  by  state,  is  an  important  compo¬ 
nent  used  in  econometric  models  of  the  oil  and  gas  industry.  Such 
econometric  models  are  of  interest  to  the  U.S.  Department  of  Energy. 


*This  data  and  its  description  was  given  to  us  by  Mrs.  B.  Volpe,  Energy 
Information  Office,  U.S.  Department  of  Energy,  Washington,  D.C.  It 
has  been  abstracted  from  a  Hughes  Tool  Company,  Houston,  Texas,  report, 
entitled  "Average  Number  of  Rotary  Rigs  Running  -  by  State." 
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In  Table  8.11,  we  give  82  values  of  the  weekly  number  of  rotary 
rigs  in  use  in  the  Southern  Louisiana  Inland  Waterways,  from  the  period 
starting  December  10,  1979.  A  time  series  plot  of  this  data  is  shown  in 
Figure  8.24.  An  informal  inspection  of  this  plot  reveals  that  the  neigh¬ 
boring  observations  tend  to  behave  similarly,  suggesting  a  positive 
correlation  between  them.  This  is  in  contrast  to  neighboring  values 
alternating  in  sign  implying  a  negative  correlation  between  them.  There 
does  not  appear  to  be  a  well  discernable  underlying  trend  in  this  data, 
nor  does  the  data  reveal  any  systematic  fluctuations  indicative  of  an 
underlying  periodicity. 

In  Tables  8.12  and  8.13,  we  give  values  of  the  estimated  autocorre¬ 
lations  and  partial  autocorrelations  of  this  data  for  lags  1,2,. ..,25. 

In  Figures  8.25  and  8.26,  we  show  plots  of  the  estimated  autocorrelation 
function  and  partial  autocorrel ation  function,  respectively.  The 
estimated  autocorrelation  function  appears  to  decay  exponentially,  and  the 
estimated  partial  autocorrelation  function  shows  a  value  which  is  signi¬ 
ficantly  differently  from  zero  at  lag  1  only.  These  plots  suggest  that 
the  data  of  Table  8.11  may  be  reasonably  well  described  by  an  autoregres¬ 
sive  process  of  order  1. 

An  estimate  of  the  autoregressive  parameter  can  be  obtained  by 
using  the  estimate  r(l)  =  .722  in  (8.28).  The  estimate  r(l)  is 
consistent  with  our  observation  that  the  neighboring  values  in  Figure  8.24 
tend  to  behave  similarly. 

Based  upon  the  above,  a  proposed  linear  stochastic  model  for  the 
weekly  number  of  rotary  rigs  in  use  in  Southern  Louisiana  Inland  Waterways 
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Table  8. 11 


Values  of  the  Weekly  Rotary  Rigs  in  Use  in 
The  Southern  Louisiana  Inland  Waterways  for  the 
period  starting  December  10,  1979 
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Figure  8.24.  A  plot  of  the  Weekly  Rotary  Rigs  in  Use  in 
The  Southern  Louisiana  Inland  Waterways 
for  the  period  starting  December  10,  1979. 
(See  Table  8.11) 
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Table  8.13 
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Estimated  Partial  Autocorrelation  Estimated  Autocorrelation 
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Fiqure  8.25.  A  plot  of  the  estimated  autocorrelation 
function  of  the  Weekly  Rigs  in  Use  data. 


Figure  8.26.  A  plot  of  the  estimated  partial  autocorre¬ 
lation  function  of  the  Weekly  Rigs  in  Use 
data. 
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y  is  for  t  =  2, . . . ,82 

(yt  -  78.5)  =  .722(yt_1 -78.5) +Gt  , 
where  the  u^'s  ,  with 

ut  =(yt-  78.5)  -  .722(yt_1- 78.5)  ,  t  =  2,...,82  , 

are  known  as  the  residuals.  The  quantity  78.5  represents  the  mean  of 
the  data. 

In  order  to  assess  how  well  the  proposed  mdoel  describes  the  data 
of  Table  8.11,  we  see  if  there  is  any  recognizable  pattern  in  the  resid¬ 
uals.  If  the  model  were  adequate,  then  we  would  expect  that  as  the 
series  length  increases,  the  u^'s  would  become  close  to  the  innovations 

A 

u^'s  .  Thus  a  study  of  the  u^'s  would  indicate  the  existence  and 
possibly  the  nature  of  model  inadequacy.  In  particular,  the  behavior  of 

A 

the  estimated  autocorrelation  and  partial  autocorrelation  of  the  u^'s 
would  yield  valuable  evidence  about  model  inadequacy.  The  absence  of  a 
recognizable  pattern  in  a  plot  of  these  functions  would  give  us  some 
assurance  of  model  adequacy. 

In  Tables  8.14  and  8.15,  we  give  values  of  the  estimated  autocorre¬ 
lations  and  partial  autocorrelations  of  the  residuals  in  question  for 
lags  1,...,25.  In  Figures  8.27  and  8.28,  we  show  plots  of  the  data  in 
Tables  8.14  and  8.15.  Since  these  plots  do  not  reveal  any  recognizable 
pattern,  we  conclude  the  adequacy  of  the  proposed  model. 


Table  8.15 
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Figure  8.27.  A  plot  of  the  estimated  autocorrelation  function 
of  the  residuals  from  an  AR(1)  model  for  the 
Weekly  Rigs  in  Use  data. 


s 


Figure  8.28.  A  plot  of  the  estimated  partial  autocorrelation 
function  of  the  residuals  from  an  AR(1)  model 
for  the  Weekly  Rigs  in  Use  data. 
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8.11.2  The  Landsat  2  Satellite  Data* 

The  Landsat  2  satellite  is  an  earth  orbiting  satellite  which 
measures  the  amount  of  reflected  energy  in  4  bands  of  the  electromag¬ 
netic  spectrum.  The  satellite  travels  from  north  to  south  over  the 
day  side  of  the  earth.  As  the  satellite  travels,  an  oscillating 
mirror  sweeps  out  a  150  kilometer  long  scan  line  in  a  west  to  east 
direction.  The  mirror  reflects  the  energy  from  the  ground  onto  an 
array  of  detectors  on  board  the  satellite.  The  reflected  energy  is 
converted  to  an  electrical  impulse.  Several  such  impulses  are  inte¬ 
grated  over  a  short  period  of  time  and  then  transmitted  to  the  earth 
via  ground  stations  as  discrete  signals.  The  discrete  signals 
represent  light  intensities,  0  denoting  black,  and  large  values 
such  as  130  denoting  bright. 

In  Table  8.16  we  give  496  values  of  the  light  intensities 
observed  by  such  a  satellite  over  a  sand  dune  field  in  the  Sahara 
Desert.  The  measurements  are  indexed  by  the  distance  traveled  by  the 
satellite  (instead  of  time)  and  are  recorded  at  every  80  meter 
distance.  Note  that  the  entries  in  Table  8.16  are  taken  from  a 
computer  output  with  an  exponent  notation;  thus  the  first  observation 
.740000  +  02  denotes  74.000  .  The  light  intensities  in  this  table 
range  from  0  to  127  .  In  Figure  8.29  we  show  a  plot  of  the  first 


*This  data  and  its  description  was  given  to  us  by  Dr.  Mark  Labovitz  of 
the  National  Aeronautics  and  Space  Administration,  at  the  Goddard 
Space  Flight  Center,  Greenbelt,  Maryland  20771. 
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Table  8.16 


Values  of  the  Light  Intensities  Observed  by  a 
Landsat  2  Satellite  over  the  Sahara  Desert 


1-  8 

0.740000+02  .0.710000+02 

t-  16 

0.660000+02 

0 . 630000+02 

17-  24 

0. 860000+02 

0.790000+02 

25-  32 

0.830000+02 

0.780000+02 

33-  40 

0.940000+02 

0.101000+03 

41-  48 

0 . 95000D+02 

0.102000+03 

49-  56 

0. 530000+02 

0.7  00000  +  02 

57-  64 

0. 380000+02 

0.950000+02 

65-  72 

0. 78COO0tO2 

0 , 77000D+02 

73-  30 

0.105000+03 

0.103000+03 

Ui-  88 

0.870000+02 

0 .860000+02 

89-  96 

0 .105000+03 

0- 1O1OO0-O3 

97-  104 

0. 760000+02 

0 .740000+02 

105-  112 

0.770000+02 

0.340000+02 

11 3-  1^0 

O.72Ovc0+O2 

0 . 630000+02 

121-  128 

0.920000+02 

0.900000+02 

129—  136 

0. 900000+02 

V  ■  v:,. >008+02 

l“37-  14-1 

0 . 870000+02 

0,910000+02 

145-  152 

0 .960000+02 

0.950000+02 

153-  160 

0,800008+02 

0 . 730000+02 

161  —  ic>8 

0 . 920000+02 

0 . 390000+02 

169-  176 

0.900000+02 

0.850000+02 

177-  184 

0.790000+02 

0.820000+02 

185-  192 

0. 600000+02 

0.640000+02 

1 ?  3-  200 

0.640000+02 

O.O7000D+02 

201-  20 R 

0.370000+0? 

0. 870000+02 

209-  2J  6 

0 .740000+02 

0 . 670000+02' 

217-  2i<! 

0.671000+02 

o.  6300004-02 

225-  222 

0.970000+02 

0.870000+02 

233-  240 

0.620000+02 

0.630000+02 

241-  248 

0.620000+02 

0.590000+02 

249-  25c 

0.300000+02 

•9  .370000+02’ 

257-  264 

0.330000+02 

0 .770000+02 

265-  272 

0.340000+02 

0.83000D+02 

273-  280 

0.610000+02 

0. 670000+02 

>281.-  288 

0  *  67G00B+02 

0 . 8900CP+02 

296 

0.970000+02 

0-.  ?“0OO0+OZ 

29  7  -  504 

0 .990000+02 

0  -  i  O'  x  0  v  £:  -f  0  3 

Z9~‘~  "5 1 Z 

0.300000+02 

0 . 830000+02 

"  \  Z  -  7 0 

0. T 3 0000+0? 

0. 106000+03 

^21  -  322 

0.760000+02 

0 .800000+02 

0 ♦ 63000D+02 

0  .  ■'i  fOC.00+02' 

J  i  7  -  7  '  i  / 

0.750000+02 

•J  *  660000+02 

0. 7300004-02 

0.740000+02 

353-  360 

0.690000+02 

0 ♦ 6S000D+02 

Joi-  2>6S 

0.660000+02 

0.690000+02 

c6r~  3/6 

0. 910000+02 

0 .  X  '20CD+03l 

277-  224 

0.330000+02 

o . 75000D+02 

iS5-  3 92 

0 . 9 80000+02 

0. 910000+02 

392-  400 

0 . 520000+02 

0.540000+02 

-«ji-  408 

0.800000+02 

0 , 500000+02 

AO'?-  416 

0  370000+02 

41/-  424 

0.710000+02 

0.75/0 00+02 

425-  432 

0, 111000+03 

0* L050CB+03 

173-  440 

0 . 380000+02 

?.  900000+02 

4+1-  -14  3 

0 ♦ 9  GOOGDt  0  2 

0.340000+02 

■4-4-;  - 

0.950000+02 

0 .840000+02 

Zi  /  '40"T 

’,/*  •  VjvOCU  »' 

v. 9 60000+02 

465-  4/2 

Ki  ♦  C/0  L* 

■j  .  8100004-02 

*4/3-  -4«0 

0 .820000+02 

J+  / 

4 Si-  488 

0 .  V8OOO0+O- 

0 .940000+02 

48V-  49c 

0.910000+02 

'  0  .£80000+02 

0.740000+02 
O.7OOOO0+O2 
0.750000+02 
0.860000+02 
0 .110000+03 
0.101000+03 
0.800000+02 
0.990000+02 
0.730000+02 
0.100000+03 
0. 900000+02 
0.900000+02 
0.750000+02 
0.760000+02 
0_.  730000+02 
0.890000+02 
0 . 930000+02 
0.910000+02 
0.960000+02 
0.700000+02 
0.840000+02 
0.860000+02 
0.330000+02 
0.720000+02 
0.730000+02 
0.900000+02 
0.780000+02 
0. 690000+02 
0.390000+02 
0.590000+02 
0.360000+02 
0-910000+02 
0.710000+02 
0. 820000+02 
0.790000+02 
0.830000+02 
0.920000+02 
0-990000+02 
0,930000+02 
0. 1C800D+C3 
0 . 730000+02 
0-710C0D+02 
0. 810000+02 
0.630000+02 
0. 690000+02 
0.710000+02 
0.100000+03 
0 • 73000D+02 
0.340000+02 
0.610000+02 
0.830000+02 
0 • 890000+02 
0.910000+02 
0 . 950000+02 
0.960000+02 
0. 820000+02 
0.790000+02 
0.920000+02 
0.91GOCD+02 
0. 380000+02 
0.950000+0* 
0 .240000+02 


0,760000+02 
0.860000+02 
0.710000+02) 
0.930000+02 
0.111000+03 
0.35000D+02 
0.780000+02 
0.970000+02 
0. 750000+02 
0.100000+03 
0.101000+03 
0. 790000+02 
0. 330000+02 
0.720000+02 
0.790000+02 
0.860000+02 
0  -  360000+02 
0.330000+02' 
0. 930000+02 
0.770000+02 
0.840000+02 
0.330000+02 
0. 79000 0+0? 
0. 740000+02 
0.790000+02 
0.960000+02 
0.340000+02 
0.720000+0? 
0.830000+02 
0.630000+02 
0.620000+02 
0. 100000+03 
0.610000+021 
O.S3000D+02 
0.870000+02 
0 . S60C00+02 
0. 910000+02 
0.103000+03- 
0. 970000+02 
0.110000+03 
0.690000+02 
0 . 660000+02 
0 . 35000D+021 
0. 660000+02 
0.620000+02 
C- 740000+02 
0.960000+02 
0.750000+021 
0.3^0000+02 
0.780000+02 
0.340000+02 
0-270000+02 
0-102000+031 
0.102000+03 
0.930000+02 
0.770000+02 
0.800000+02 
0.950000+03 
0.82oO0D+02 
o .  ■/"  wOO01)+0-^ 
O  »feuGOO04*o,l 
0.690000+02 


0.680000+02 

0.940000+02 

0.750000+02 

0.930000+02 
0. 102000+03 
0.780000+02 
0.730000+02 
0.930000+02 
0.770000+02 
0 . 93000D+02 
0.102000+03 
0. 700000+02 
0-370000+02 
0.720000+02 
0. 890000+02 
0 . 770000+02 
0.820000+02 
0.310000+02 
0.830000+02 
0.800000+02 
0.890000+02 
0.790000+02 
0.730000+02 
0.740000+0,2 
0.7900CD+02 
0.96C00D+02 
0.940000+02 
C. 830000+02 
0.900000+02 
0.67000D+02 
0 . 63000D+02 
0.103000+03 
0.740000+02 
0. 350000+02 
0.880000+02 
0.330000+02 
0.890000+02 
0.930000+02 
0.890000+02 
0.108000+03 
0.620000+02 
0 .650000+02" 
0.340000+02 
0.730000+02 
0. 500000+02 
0.310000+02 
0.920000+02 
0.300000+02 
0 . 320C0D+02 
0.900000+02 
0.770000+02 
0 . 96000D+02 
0. 109000+03 
0 . 99000D+02 
0.970000+02 
0.310000+02 
0.860000+02 
0 .320000+02 
0 . 79OU0U+O2 
0 . 104000+03 
0 . 7  40000+0,. 
C  .680000+02 


0.610000402 
0. 96000 D+02 
0.800000+02 

0.950000+02 
0.930000+02 
0.610000+02 
0.700000+02 
0.840000+02 
0.370000+02 
0.820000+02 
0.100000+03 
0. 670000+02 
0.870000+02 
0  *  790000+02 
C  -360COD+02 
0 . 75000D+02 
0.77000D+02 
0.840000+02 
0.760000+02 
0.840000+02 
0.950000+02 
0.770000+02 
0.690000+02 
0.69000 0+02 
0.850000+02 
0.101000+03 
0.960000+02 
0.840000+02 
0 . 920C00+02 
0.67CCC0+02 
0.710000+02 
0.930000+02 
0. 370^00+02 
0. 340000+02 
0. 330000+02 
0. 340000+02 
0.930000+02 
0 .830000+02 
0.  0000+02' 
0.900000+02 
0.620000+02 
0. 690000+02 
0.850000+02 
<0, 340000+02'. 
0.320000+02 
0. 840000+02 
0.960000+02 
0.850000+02 
(0 .7- 0000+02 
0-990000+02 
0 . 33000D+02 
0.94000D+02 
0 . 10700D+03 
0.890000+02 
0.99000D+02 
0. 380000+02 
0.900000+02 
0.740000+02 
0 . 750000+02 
w.  102000+03 
0.750000+02 
0.760000+02 


0.58000D+02 
0.96000D+02 
0.82000D+02 
0. 860000+02 
0 . 960000+0? 
0.540000+02 
0. 710000+02 
0.330000+02 
0.940000+02 
0.770000+02 
0.101000103 
,0.690000+02 
0.890000+02 
0.830 000+02 
0 .860000+02 
0.900000+02 
0.790000+02 
0.940000+02 
0,710000  1-02 
Z .790000+02 
0. 230000+02 
0.790000+02 
0.670000+02 
0.670000+02 
0.780000+02 
0.940000+02 
0. 900000+02 
0.910000+02 
0 .700000+02 
0.710000+0? 
0,7 4000D+02 
0.370000+02 
0.950000+02 
0.330000+02 
0. 360000+02 
0.930000+02 
0.910000+02 
0.91 0000+02 
0 .370000+02 
0. 760000+02 
0.690000+02 
0.700000+02 
0 . 330000+02 
0 . 340000+02 
0.540000+02 
0.880000+02 
0 . 94000D+C2 
0.830000+02 
0. 600000+02 
0.390000+02 
0.930000+02 
0.730000+02 
0. 11100D+03 
0.830000+02 
0.970000+02 
0.900000+02 
0.870000+02 
0.710000+02 
0.750000+02 
0.970000+C2 
0. 750000+02 
0.790000+02 


0 . 530000+02 
0, 92 000 D +62 
6.876000+62 

0. 920000+02 
0,930000+02 
0 .540000+0? 
0.710000+02 
0.790000+02 
0.101000+03 
0 . 790000+02 
0 .103000+03 
0 . 73600D+02 
0.32000D+02 
0.750000+02 
0  -  <52<%000+02 
0. 910000+02 
0.86000D+62 
0.101000+63 
0.740000+02 
0.780000+02 
0,370000+02 
C. 810000+02 
0.660000+02 
0 .620000+02 
C. 300000+02 
0.750000+02 
0.740000*02 
0.102000+03 
0-660000+02 
0 . 700000+02 
0.310000+02 
0.340000+02 
0.920000+02 
0.72C00D+02 
0 .8,10000+02 
0.96000D+02 
0.950000+02 
0.830000+02 
0.830000+02 
<&.  700000+02 
0.700000+02 
0.700000+02 
0.360000+02 
0.330000+02 
'0-620000+02 
0.380000+02 
O.S9OOO0+O2 
0  - 39000D 102 
0.580000+02 
^.340000+021 
C. 900000+02 
0 . 750000+02 
0.112000+03 
0.810000+02 
'0. 930000+02’ 
0.890000+02 
0 . 99000D+02 
0.6 40000+02 
0.740000+02 
0. 990000+02 
0.770000+02 
0.740000+02' 
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300  values  of  the  light  intensity  over  distance  traveled.  The  plotting 
of  all  the  496  observations  would  be  cumbersome  and  would  tend  to  con¬ 
ceal  the  fluctuating  behavior  of  the  individual  observations.  The 
periodic  fluctuations  in  Figure  8.29  suggest  an  autoregressive  process 
of  order  2  or  more  -  see  Section  8.6.  The  plot  also  conveys  the 
impression  of  an  underlying  trend  whose  wave  like  nature  is  indicated 
by  the  dotted  line  of  Figure  8.29.  Thus  one  possibility  would  be  first 
to  fit  a  cyclical  trend  of  the  type  described  in  Section  4.2  to  this 
data,  and  then  to  describe  the  residuals  from  such  a  fit  by  autoregres¬ 
sive  processes.  However,  this  possibility  was  not  investigated  here, 
and  instead  fluctuations  about  the  sample  mean  of  the  entire  series  of 
496  observations  were  considered.  The  sample  mean  of  the  entire  set  of 
these  data  is  82.9. 

In  Tables  8.17  and  8.18  we  give  values  of  the  estimated  autocor¬ 
relations  and  partial  autocorrelations  of  the  deviations  of  the  light 
intensities  from  82.9  for  lags  1,2,. ..,35.  In  Figures  8.30  and 
8.31  we  show  plots  of  the  estimated  autocorrelation  function  and  par¬ 
tial  autocorrelation  function,  respectively.  The  estimated  autocorre¬ 
lation  function  appears  to  decay  exponentially,  and  perhaps  even 
sinusoidally.  The  estimated  partial  autocorrelation  function  takes 
values  which  are  significantly  different  from  zero  at  lags  1  and  2. 
These  plots  suggest  that  the  deviations  of  the  light  intensities  can 
be  described  by  an  autoregressive  process  of  order  2. 

Estimates  of  the  two  autoregressive  parameters  6^  and  ^  can 
be  obtained  by  using  the  estimates  r(l)  =  .87  and  r(2)  =  .66  in 


Values  of  the  estimated  autocorrelation  function  r(h). 
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Figure  8.30.  A  plot  of  the  estimated  autocorrelation 
function  of  the  deviations  of  the  light 
intensities  from  their  mean  for  the 
Landsat  2  Satellite  data. 


Fiqure  8.31.  A  plot  of  the  estimated  partial  autocorre¬ 
lation  function  of  the  deviations  of  the 
light  intensities  from  their  mean  for  the 
Landsat  2  Satellite  data. 
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Equation  (8.28);  these  turn  out  to  be  =  -1.28  and  b2  =  .46  .  The 
estimate  of  82  1S  approximately  the  estimated  value  of  the  partial 
autocorrelation  at  lag  2,  which  is  -.40  . 

Based  upon  the  above,  a  proposed  linear  stochastic  model  for  the 
light  intensity  yt  ,  is,  for  t  =  3,...,496,  (y^  - 82.9)  =  1.28(yt_^-82.9) 
-  .46(yt_2  -  82.9)  + u^  ,  where  the  u^'s  ,  with 

ut  =  (yt-82.9)  -  1.28(yt_1-82.9)  +  .46(yt_2-82.9)  ,  t  =  3, . . . ,496  , 
are  known  as  the  residual s . 

In  order  to  judge  how  well  the  proposed  model  describes  the  data 
of  Table  8.16,  we  see  if  there  is  any  recognizable  pattern  in  the 
residuals.  If  the  model  were  adequate,  then  we  would  expect  that  as 

A 

the  series  length  increases  the  u^'s  would  become  close  to  the 
innovations  u^'s  .  Thus  a  study  of  the  u^'s  would  indicate  the 
existence  and  possibly  the  nature  of  model  inadequacy.  In  particular, 
the  behavior  of  the  estimated  autocorrelation  and  partial  autocorrela- 

A 

tion  functions  of  the  u^'s  would  yield  valuable  evidence  about  model 
inadequacy.  The  absence  of  a  recognizable  pattern  in  a  plot  of  these 
functions  would  give  us  some  assurance  of  model  adequacy. 

In  Tables  8.19  and  8.20  we  give  values  of  the  estimated  autocor¬ 
relations  and  partial  autocorrelations  of  the  residuals  in  question 
for  lags  1,2,..., 35  .  In  Figures  8.32  and  8.33,  we  show  plots  of  the 
data  in  Tables  8.14  and  8.15,  respecti vely.  Since  these  plots  do  not 

show  any  recognizable  pattern,  we  conclude  that  the  proposed  model  is 
adequate. 


Values  of  the  estimated  autocorrelation  function  r(h), 
h = 1,2, . . . ,35,  of  the  residuals  from  an  AR(2)  model 


Values  of  the  estimated  partial  autocorrelation  function  n(h) 
h  =  l,2 . 35,  of  the  residuals  from  an  AR(2)  model 
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Figure  8.32. 


Figure  8.33.  A  plot  of  the  est 

. .  function  of  the  r 

for  the  Landsat  2 
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